Jul 06 23:54:30.775345 ip-10-0-129-135 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 23:54:30.775354 ip-10-0-129-135 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 23:54:30.775360 ip-10-0-129-135 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 23:54:30.775581 ip-10-0-129-135 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 23:54:40.868133 ip-10-0-129-135 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 23:54:40.868152 ip-10-0-129-135 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5d2ac732e6ee41b78b04db80b2c3db96 --
Jul 06 23:56:35.956411 ip-10-0-129-135 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 23:56:38.320144 ip-10-0-129-135 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:38.320144 ip-10-0-129-135 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 23:56:38.320144 ip-10-0-129-135 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:38.320144 ip-10-0-129-135 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 23:56:38.320144 ip-10-0-129-135 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:38.320871 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.320771    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 23:56:38.324541 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324526    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.324541 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324541    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324545    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324548    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324551    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324554    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324556    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324559    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324563    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324566    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324569    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324571    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324574    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324577    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324579    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324582    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324584    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324587    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324589    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324592    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324595    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.324594 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324598    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324601    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324605    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324608    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324610    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324613    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324616    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324619    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324621    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324624    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324626    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324629    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324631    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324634    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324636    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324639    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324641    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324644    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324647    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324649    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.325075 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324651    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324654    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324657    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324659    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324661    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324664    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324666    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324669    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324671    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324673    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324676    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324678    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324681    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324683    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324686    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324689    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324692    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324694    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324697    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324699    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.325542 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324702    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324704    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324707    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324710    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324713    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324717    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324721    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324724    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324727    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324730    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324733    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324737    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324740    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324743    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324745    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324748    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324750    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324753    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.326049 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324755    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324757    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324760    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324764    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324766    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324769    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324771    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324775    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324780    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324783    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324786    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324789    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324792    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324794    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324796    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324799    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324802    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324805    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324807    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.326467 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.324810    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325508    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325515    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325518    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325521    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325524    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325529    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325533    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325536    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325539    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325542    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325545    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325548    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325550    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325553    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325556    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325559    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325561    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325564    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.326979 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325566    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325569    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325572    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325575    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325578    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325581    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325583    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325586    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325588    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325591    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325593    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325596    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325599    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325601    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325604    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325606    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325608    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325611    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325613    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325616    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.327431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325619    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325621    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325624    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325626    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325629    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325631    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325633    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325636    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325638    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325642    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325645    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325648    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325651    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325654    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325657    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325659    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325662    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325665    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325668    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325670    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.327952 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325673    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325675    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325678    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325681    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325683    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325686    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325688    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325691    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325694    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325696    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325699    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325701    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325704    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325706    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325709    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325711    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325714    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325716    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325719    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325721    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.328431 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325724    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325726    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325728    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325732    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325735    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325738    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325741    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325744    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325746    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325749    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325751    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325754    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325756    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325759    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325761    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325765    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325768    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325770    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325773    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325776    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.328926 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.325778    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326157    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326162    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326165    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326167    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326170    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326173    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326176    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326178    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326181    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326184    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326187    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326189    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326193    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326198    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326201    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326203    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326207    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326210    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.329479 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326212    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326216    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326220    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326224    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326227    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326230    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326232    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326235    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326238    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326241    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326243    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326246    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326248    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326251    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326253    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326256    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326258    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326261    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326263    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.330021 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326266    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326270    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326272    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326275    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326277    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326280    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326282    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326286    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326289    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326292    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326295    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326298    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326301    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326303    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326306    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326308    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326311    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326314    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326317    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326319    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.330530 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326323    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326326    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326328    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326331    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326333    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326336    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326338    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326341    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326343    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326346    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326348    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326351    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326353    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326356    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326359    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326361    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326363    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326366    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326368    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326371    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.331096 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326373    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326376    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326378    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326381    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326384    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326386    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326389    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326391    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326394    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326396    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326399    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326401    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326404    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326406    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326408    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326411    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326413    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326416    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326418    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326421    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.331632 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326423    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.326426    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326498    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326505    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326511    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326515    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326520    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326523    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326527    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326531    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326534    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326538    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326543    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326546    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326550    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326553    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326555    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326558    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326561    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326564    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326569    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326572    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326575    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326578    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 23:56:38.332199 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326582    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326585    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326588    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326591    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326594    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326597    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326600    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326603    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326606    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326609    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326613    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326617    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326620    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326622    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326625    2581 flags.go:64] FLAG: --enable-server="true"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326629    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326633    2581 flags.go:64] FLAG: --event-burst="100"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326636    2581 flags.go:64] FLAG: --event-qps="50"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326639    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326642    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326646    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326650    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326653    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326656    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326659    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 06 23:56:38.332842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326663    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326666    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326669    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326671    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326674    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326677    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326680    2581 flags.go:64] FLAG: --feature-gates=""
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326683    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326686    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326689    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326692    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326695    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326698    2581 flags.go:64] FLAG: --help="false"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326701    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326706    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326709    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326711    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326715    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326718    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326721    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326724    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326727    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326730    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 23:56:38.333688 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326733    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326736    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326739    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326742    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326746    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326749    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326752    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326754    2581 flags.go:64] FLAG: --lock-file=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326757    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326760    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326763    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326768    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326771    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326774    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326777    2581 flags.go:64] FLAG: --logging-format="text"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326780    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326783    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326786    2581 flags.go:64] FLAG: --manifest-url=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326789    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326793    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326796    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326800    2581 flags.go:64] FLAG: --max-pods="110"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326803    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326807    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326810    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 23:56:38.334698 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326813    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326816    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326819    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326822    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326828    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326832    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326835    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326837    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326841    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326846    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326849    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326852    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326871    2581 flags.go:64] FLAG: --port="10250"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326874    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326877    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03e99713f2aa468ac"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326880    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326883    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326886    2581 flags.go:64] FLAG: --register-node="true"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326889    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326892    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326895    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326898    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326901    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326905    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326908    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 23:56:38.335751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326911    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326914    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326916    2581 flags.go:64] FLAG: --runonce="false"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326919    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326922    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326926    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326930    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326933    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326936    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326939    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326942    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326944    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326947    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326950    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326953    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326956    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326959    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326962    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326967    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326971    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326974    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326978    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326980    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326983    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326986    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 23:56:38.336526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326989    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326992    2581 flags.go:64] FLAG: --v="2"
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.326996    2581 flags.go:64] FLAG: --version="false"
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.327000    2581 flags.go:64] FLAG: --vmodule=""
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.327004    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.327007    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327117    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327121    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327125    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327128    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327131    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327135    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327140    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327144    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327147    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327150    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327152    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327155    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327158    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327160    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.337533 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327163    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327165    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327169    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327173    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327176    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327179    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327182    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327185    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327188    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327190    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327193    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327195    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327198    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327201    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327203    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327206    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327208    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327211    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327214    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.338401 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327216    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327219    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327221    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327223    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327226    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327230    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327233    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327235    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327238    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327240    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327243    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327245    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327248    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327250    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327253    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327255    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327258    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327260    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327263    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327267    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.338954 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327269    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327272    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327275    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327277    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327280    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327283    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327285    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327288    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327290    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327293    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327295    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327298    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327300    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327303    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327306    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327308    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327311    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327315    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327317    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327320    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.339733 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327322    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327325    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327328    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327331    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327333    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327336    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327338    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327341    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327343    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327345    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327348    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327352    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327354    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327357    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327360    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327362    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327365    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327367    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327370    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327372    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.340579 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327374    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327377    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327380    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327382    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327385    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.327387    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.328023    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.336032    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.336059    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.336833    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.336848    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336962    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336971    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336977    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336982    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.341155 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336987    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336992    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.336997    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337002    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337007    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337011    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337016    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337020    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337024    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337029    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337033    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337037    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337041    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337045    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337050    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337054    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337058    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337062    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337066    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.341651 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337070    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337074    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337079    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337083    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337087    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337092    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337105    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337109    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337113    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337117    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337121    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337126    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337130    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337134    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337138    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337142    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337146    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337150    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337154    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337158    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.342151 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337163    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337167    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337171    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337175    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337179    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337183    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337187    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337191    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337195    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337200    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337204    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337211    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337217    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337221    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337227    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337231    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337236    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337242    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337247    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337258    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.342670 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337263    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337268    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337272    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337276    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337280    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337285    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337289    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337293    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337297    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337301    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337305    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337309    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337314    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337318    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337322    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337326    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337330    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337334    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337348    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337352    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.343313 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337359    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337365    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337369    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337373    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337379    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337385    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337390    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337394    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337399    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337403    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337408    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337412    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337427    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337432    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337436    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.343885 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337440    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.337448    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337678    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337687    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337692    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337697    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337701    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337705    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337710    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337714    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337718    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337722    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337726    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337730    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337735    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:38.344287 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337739    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337743    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337747    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337752    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337756    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337760    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337765    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337771    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337777    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337781    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337785    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337790    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337794    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337799    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337803    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337815    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337819    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337824    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337828    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:38.344659 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337833    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337837    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337841    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337845    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337849    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337868    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337873    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337877    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337881    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337885    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337890    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337894    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337898    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337902    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337906    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337912    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337917    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337922    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337927    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:38.345174 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337931    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337935    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337939    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337943    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337948    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337952    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337956    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337961    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337966    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337972    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337984    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337989    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337993    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.337998    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338002    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338006    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338010    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338015    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338019    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338023    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:38.346005 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338027    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338031    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338035    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338039    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338044    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338048    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338052    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338056    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338060    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338064    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338068    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338072    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338076    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338081    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338084    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338088    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338093    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338097    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338101    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338105    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:38.346891 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338109    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338113    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338117    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338128    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338134    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338138    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338142    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:38.338146    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.338154    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.338385    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.341474    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.343009    2581 server.go:1019] "Starting client certificate rotation"
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.343115    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:38.347680 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.343892    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:38.348110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.345918    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:38.350959 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.350938    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:38.362157 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.362135    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 06 23:56:38.364393 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.364378    2581 log.go:25] "Validated CRI v1 image API"
Jul 06 23:56:38.365147 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.365126    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 23:56:38.367497 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.367471    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7b688ed1-07b8-461e-8566-7fd1a180c65e:/dev/nvme0n1p4 949037d3-26ee-4061-a2af-ab902e1fddba:/dev/nvme0n1p3]
Jul 06 23:56:38.367570 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.367496    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 23:56:38.372323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.372209    2581 manager.go:216] Machine: {Timestamp:2026-07-06 23:56:38.371150181 +0000 UTC m=+0.774546256 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099724 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22f1b4e675d966340aadab68a960a1 SystemUUID:ec22f1b4-e675-d966-340a-adab68a960a1 BootID:5d2ac732-e6ee-41b7-8b04-db80b2c3db96 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:00:b5:c8:58:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:00:b5:c8:58:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:1e:ab:b7:b3:23 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 23:56:38.372323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.372315    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 23:56:38.372430 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.372398    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 23:56:38.374089 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374060    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 23:56:38.374235 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374093    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-135.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 23:56:38.374324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374308    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:38.374923 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374913    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 23:56:38.374956 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374926    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 23:56:38.374956 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.374934    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:38.376252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.376219    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:38.377822 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.377811    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 23:56:38.381037 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381026    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:38.381154 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381145    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 23:56:38.381182 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381173    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 23:56:38.381210 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381183    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 23:56:38.381210 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381196    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 23:56:38.381210 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381205    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 06 23:56:38.381312 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381216    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 23:56:38.381604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381593    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:38.381643 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.381612    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:38.383673 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.383649    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 23:56:38.385777 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.385763    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 23:56:38.385840 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.385792    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 23:56:38.387250 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387237    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387253    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387259    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387264    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387273    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387278    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387287    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387292    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387298    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387303    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387312    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 23:56:38.387323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.387317    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 23:56:38.388128 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.388117    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 23:56:38.388128 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.388127    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 23:56:38.391082 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.391057    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-135.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 23:56:38.391271 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.391252    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 23:56:38.391447 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.391429    2581 server.go:1268] "Started kubelet"
Jul 06 23:56:38.391545 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.391514    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 23:56:38.391599 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.391525    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 23:56:38.391654 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.391626    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 23:56:38.391845 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.391825    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-135.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 23:56:38.392367 ip-10-0-129-135 systemd[1]: Started Kubernetes Kubelet.
Jul 06 23:56:38.393548 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.393525    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 23:56:38.395102 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.395083    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 23:56:38.398208 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.398186    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 23:56:38.400473 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.399615    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-135.ec2.internal.18bfd8a921dfd3a4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-135.ec2.internal,UID:ip-10-0-129-135.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-135.ec2.internal,},FirstTimestamp:2026-07-06 23:56:38.391395236 +0000 UTC m=+0.794791301,LastTimestamp:2026-07-06 23:56:38.391395236 +0000 UTC m=+0.794791301,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-135.ec2.internal,}"
Jul 06 23:56:38.401485 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.401469    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zfmpl"
Jul 06 23:56:38.402278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.402260    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 23:56:38.402278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.402276    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:38.402407 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.402332    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.403036 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403019    2581 factory.go:55] Registering systemd factory
Jul 06 23:56:38.403116 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403095    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 06 23:56:38.403116 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403030    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 23:56:38.403226 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403055    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 23:56:38.403226 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403155    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 23:56:38.403313 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403270    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 23:56:38.403313 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403279    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 23:56:38.403418 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403342    2581 factory.go:153] Registering CRI-O factory
Jul 06 23:56:38.403418 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403354    2581 factory.go:223] Registration of the crio container factory successfully
Jul 06 23:56:38.403418 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403401    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 23:56:38.403572 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403423    2581 factory.go:103] Registering Raw factory
Jul 06 23:56:38.403572 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403438    2581 manager.go:1196] Started watching for new ooms in manager
Jul 06 23:56:38.403760 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.403749    2581 manager.go:349] Starting recovery of all containers
Jul 06 23:56:38.408527 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.408363    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zfmpl"
Jul 06 23:56:38.408702 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.408675    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-135.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 23:56:38.409190 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.409169    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 23:56:38.413811 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.413798    2581 manager.go:354] Recovery completed
Jul 06 23:56:38.418142 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.418129    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.419829 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.419814    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.419906 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.419842    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.419906 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.419870    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.420413 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.420398    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 23:56:38.420413 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.420412    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 23:56:38.420512 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.420426    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.420512 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.420430    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:38.421557 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421537    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.421624 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421580    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.421624 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421592    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.421624 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421607    2581 policy_none.go:49] "None policy: Start"
Jul 06 23:56:38.421759 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421626    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 23:56:38.421759 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.421642    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 23:56:38.422840 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.422824    2581 policy_none.go:47] "Start"
Jul 06 23:56:38.466391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466371    2581 manager.go:337] "Starting Device Plugin manager"
Jul 06 23:56:38.466453 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.466421    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 23:56:38.466453 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466433    2581 server.go:87] "Starting device plugin registration server"
Jul 06 23:56:38.466668 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466657    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 23:56:38.466700 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466670    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 23:56:38.466761 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466746    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 23:56:38.466841 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466831    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 23:56:38.466911 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.466844    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 23:56:38.467403 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.467382    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 23:56:38.467465 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.467432    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.550153 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.550111    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 23:56:38.551409 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.551392    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 23:56:38.551495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.551430    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 23:56:38.551495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.551456    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 23:56:38.551495 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.551489    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 23:56:38.553784 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.553765    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:38.567579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.567560    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.568528 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.568512    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.568584 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.568546    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.568584 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.568561    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.568584 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.568584    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.576599 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.576584    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.576684 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.576611    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-135.ec2.internal\": node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.586399 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.586376    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.652089 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.652010    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"]
Jul 06 23:56:38.652209 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.652095    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.653060 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.653043    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.653145 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.653079    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.653145 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.653094    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.654446 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.654431    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.654558 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.654542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.654607 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.654570    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.655180 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655162    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.655180 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655172    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.655288 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655190    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.655288 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655193    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.655288 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655200    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.655288 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.655204    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.656426 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.656409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.656487 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.656439    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:38.657133 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.657116    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:38.657200 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.657148    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:38.657200 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.657162    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:38.680558 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.680540    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-135.ec2.internal\" not found" node="ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.685244 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.685228    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-135.ec2.internal\" not found" node="ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.687278 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.687263    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.705335 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.705315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.705444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.705352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.705444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.705379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/200bd357d82b41d3210c766efd1f8f3e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-135.ec2.internal\" (UID: \"200bd357d82b41d3210c766efd1f8f3e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.787362 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.787333    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.805777 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.805834 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.805834 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/200bd357d82b41d3210c766efd1f8f3e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-135.ec2.internal\" (UID: \"200bd357d82b41d3210c766efd1f8f3e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.805951 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805836    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/200bd357d82b41d3210c766efd1f8f3e-config\") pod \"kube-apiserver-proxy-ip-10-0-129-135.ec2.internal\" (UID: \"200bd357d82b41d3210c766efd1f8f3e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.805951 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805880    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.805951 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.805893    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed96054e679b6790a225dcd3f67a3fd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal\" (UID: \"ed96054e679b6790a225dcd3f67a3fd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.888471 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.888422    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.984042 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.983974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:38.989099 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:38.989078    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:38.989099 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:38.989078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:39.089206 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:39.089153    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:39.189666 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:39.189636    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:39.290225 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:39.290144    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:39.342643 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.342618    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 23:56:39.343296 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.342767    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:56:39.390384 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:39.390350    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:39.412120 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.412084    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:51:38 +0000 UTC" deadline="2028-03-22 17:11:49.230237388 +0000 UTC"
Jul 06 23:56:39.412120 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.412111    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14993h15m9.818130642s"
Jul 06 23:56:39.491521 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:39.491483    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-135.ec2.internal\" not found"
Jul 06 23:56:39.536898 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.536868    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:39.551301 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.551228    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:39.587013 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.586986    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:39.602757 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.602730    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:39.613565 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.613548    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:39.614452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.614439    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal"
Jul 06 23:56:39.618708 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:39.618684    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200bd357d82b41d3210c766efd1f8f3e.slice/crio-590b1b37819781cdcf27bbdb06c4ae54b2504d0aee1b7323d6270c048bced9e3 WatchSource:0}: Error finding container 590b1b37819781cdcf27bbdb06c4ae54b2504d0aee1b7323d6270c048bced9e3: Status 404 returned error can't find the container with id 590b1b37819781cdcf27bbdb06c4ae54b2504d0aee1b7323d6270c048bced9e3
Jul 06 23:56:39.619222 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:39.619208    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded96054e679b6790a225dcd3f67a3fd9.slice/crio-3dacd67ac1acf84db5168a998df8a3f6d4b7804388cdb10bb7a708b7ebf8af65 WatchSource:0}: Error finding container 3dacd67ac1acf84db5168a998df8a3f6d4b7804388cdb10bb7a708b7ebf8af65: Status 404 returned error can't find the container with id 3dacd67ac1acf84db5168a998df8a3f6d4b7804388cdb10bb7a708b7ebf8af65
Jul 06 23:56:39.624206 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.624187    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:39.625248 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:39.625226    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:56:40.235908 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.235872    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:40.383063 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.382771    2581 apiserver.go:52] "Watching apiserver"
Jul 06 23:56:40.391178 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.391122    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 23:56:40.391530 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.391509    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-xfjnl","kube-system/global-pull-secret-syncer-hp5lp","openshift-cluster-node-tuning-operator/tuned-6dhv9","openshift-image-registry/node-ca-kxqwg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal","openshift-multus/multus-additional-cni-plugins-dpx2n","openshift-multus/multus-mql6s","openshift-ovn-kubernetes/ovnkube-node-slnr9","kube-system/konnectivity-agent-hf98m","kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh","openshift-dns/node-resolver-ls9cb","openshift-multus/network-metrics-daemon-s7xxc","openshift-network-diagnostics/network-check-target-6llz8"]
Jul 06 23:56:40.392825 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.392800    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.394287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.394266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.395225 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.395204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.395521 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.395500    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rn9p4\""
Jul 06 23:56:40.395811 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.395793    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.395948 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.395932    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 23:56:40.396056 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.396037    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.396160 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.396145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 23:56:40.396333 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.396319    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.396933 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.396915    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.397145 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.397262 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.396915    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 23:56:40.397387 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397368    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xcf5v\""
Jul 06 23:56:40.397762 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qt4c8\""
Jul 06 23:56:40.397762 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397618    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.397762 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:40.397762 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.397750    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:40.398036 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.397902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.399606 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.399568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.400404 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.400124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.400404 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.400192    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gxhrf\""
Jul 06 23:56:40.400404 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.400301    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.400404 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.400307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 23:56:40.401252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.401206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.402075 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.402033    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 23:56:40.402075 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.402049    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 23:56:40.402344 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.402259    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96dd5\""
Jul 06 23:56:40.403664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.402687    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:40.404606 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.404582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-thv5f\""
Jul 06 23:56:40.404699 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.404617    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.404699 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.404690    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:40.406912 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.405078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.406912 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.405120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 23:56:40.406912 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.405157    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:40.406912 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.405467    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 23:56:40.407497 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.407288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.409213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.409192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.410218 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.410197    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 23:56:40.410554 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.410533    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.410729 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.410708    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.410792 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.410767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xchlh\""
Jul 06 23:56:40.411129 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.410994    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 23:56:40.411310 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.411293    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 23:56:40.411543 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.411524    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 23:56:40.411798 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.411777    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.411798 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.411790    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 23:56:40.412096 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412080    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.412521 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vl48l\""
Jul 06 23:56:40.412611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-system-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.412611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysconfig\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.412611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-socket-dir-parent\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdcs\" (UniqueName: \"kubernetes.io/projected/51e3d0e3-d802-42cb-89e8-29aed5331d2f-kube-api-access-mxdcs\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5txm\" (UniqueName: \"kubernetes.io/projected/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kube-api-access-m5txm\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pjmv\" (UniqueName: \"kubernetes.io/projected/a06388ff-e2be-4364-ba0f-2d0b942e2243-kube-api-access-8pjmv\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-lib-modules\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.412768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-os-release\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-bin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-device-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-k8s-cni-cncf-io\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-socket-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.412981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-os-release\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-registration-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-kubernetes\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-host\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-cnibin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-kubelet\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-conf-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-conf\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-netns\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-sys-fs\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-sys\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-var-lib-kubelet\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-hostroot\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-tmp\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/914c2087-5c4e-4e81-a3eb-f573745db5b4-serviceca\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e867e8f-9177-4f92-a6c7-e64255669905-agent-certs\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.413618 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-tuned\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cnibin\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2867\" (UniqueName: \"kubernetes.io/projected/704d581b-123b-42b3-a7d7-176251a58865-kube-api-access-z2867\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-multus\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-modprobe-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/914c2087-5c4e-4e81-a3eb-f573745db5b4-host\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e867e8f-9177-4f92-a6c7-e64255669905-konnectivity-ca\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-kubelet-config\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-dbus\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-run\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-cni-binary-copy\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.413992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-daemon-config\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgpf\" (UniqueName: \"kubernetes.io/projected/914c2087-5c4e-4e81-a3eb-f573745db5b4-kube-api-access-gkgpf\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414072    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-system-cni-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-etc-kubernetes\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.414213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-systemd\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-multus-certs\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.414983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6xk\" (UniqueName: \"kubernetes.io/projected/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-kube-api-access-ht6xk\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.414983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.414241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.415338 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.415318    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:40.416223 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.416020    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7jsw\""
Jul 06 23:56:40.416223 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.416120    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 23:56:40.416223 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.416215    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 23:56:40.442717 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.442691    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddgxd"
Jul 06 23:56:40.450167 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.450140    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddgxd"
Jul 06 23:56:40.504607 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.504529    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 23:56:40.514987 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.514956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-netns\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.514998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-sys-fs\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-sys\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-var-lib-kubelet\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515083    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-netns\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b670d367-749c-4759-863b-02bd013fc4cf-iptables-alerter-script\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.515138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8slvz\" (UniqueName: \"kubernetes.io/projected/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-kube-api-access-8slvz\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515197    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-sys-fs\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-hostroot\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515275    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-tmp\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/914c2087-5c4e-4e81-a3eb-f573745db5b4-serviceca\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515310    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-var-lib-kubelet\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e867e8f-9177-4f92-a6c7-e64255669905-agent-certs\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515384    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-sys\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-tuned\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cnibin\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515494    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-hostroot\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-systemd-units\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515539    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cnibin\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.515590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-slash\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-node-log\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-log-socket\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs57w\" (UniqueName: \"kubernetes.io/projected/b670d367-749c-4759-863b-02bd013fc4cf-kube-api-access-rs57w\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515781    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515805    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/914c2087-5c4e-4e81-a3eb-f573745db5b4-serviceca\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2867\" (UniqueName: \"kubernetes.io/projected/704d581b-123b-42b3-a7d7-176251a58865-kube-api-access-z2867\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-netns\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-multus\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.515968    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.515978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-modprobe-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.516051    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.016013087 +0000 UTC m=+3.419409154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/914c2087-5c4e-4e81-a3eb-f573745db5b4-host\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e867e8f-9177-4f92-a6c7-e64255669905-konnectivity-ca\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-kubelet-config\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516141    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-modprobe-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.516495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516334    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-multus\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-dbus\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-etc-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-run\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-cni-binary-copy\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516464    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516504    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-kubelet-config\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-daemon-config\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgpf\" (UniqueName: \"kubernetes.io/projected/914c2087-5c4e-4e81-a3eb-f573745db5b4-kube-api-access-gkgpf\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516539    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/914c2087-5c4e-4e81-a3eb-f573745db5b4-host\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-system-cni-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-etc-kubernetes\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-systemd\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-kubelet\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjcx\" (UniqueName: \"kubernetes.io/projected/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-kube-api-access-lsjcx\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516895    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-cni-binary-copy\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-multus-certs\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.517425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516938    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-run\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.516975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6xk\" (UniqueName: \"kubernetes.io/projected/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-kube-api-access-ht6xk\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-system-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysconfig\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-systemd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-script-lib\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517202    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-dbus\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-tmp-dir\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-socket-dir-parent\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517256    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-multus-certs\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdcs\" (UniqueName: \"kubernetes.io/projected/51e3d0e3-d802-42cb-89e8-29aed5331d2f-kube-api-access-mxdcs\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517305    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-system-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517312    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-daemon-config\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.518620 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5txm\" (UniqueName: \"kubernetes.io/projected/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kube-api-access-m5txm\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pjmv\" (UniqueName: \"kubernetes.io/projected/a06388ff-e2be-4364-ba0f-2d0b942e2243-kube-api-access-8pjmv\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-lib-modules\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-netd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-config\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-env-overrides\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517605    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-d\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517611    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovn-node-metrics-cert\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-os-release\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-bin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517797    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-cni-bin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-device-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.519475 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-bin\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-k8s-cni-cncf-io\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-socket-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-var-lib-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.517987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-ovn\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b670d367-749c-4759-863b-02bd013fc4cf-host-slash\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-hosts-file\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-os-release\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-registration-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518141    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-kubernetes\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-host\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518194    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-kubernetes\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-cnibin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-kubelet\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518267    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-lib-modules\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.520363 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-conf-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-conf\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518306    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-conf-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518443    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-os-release\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.518434    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-socket-dir-parent\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.520636    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e867e8f-9177-4f92-a6c7-e64255669905-konnectivity-ca\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.520954    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521025    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-os-release\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521110 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521076    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysconfig\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521119    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-run-k8s-cni-cncf-io\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521136    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521221    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-socket-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.521238    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.521298    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.021274825 +0000 UTC m=+3.424670881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521310    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-registration-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521376    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-multus-cni-dir\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521433    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-host\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521477    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-etc-kubernetes\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.521525 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521519    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51e3d0e3-d802-42cb-89e8-29aed5331d2f-system-cni-dir\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.521981 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521565    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-systemd\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.521981 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521615    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.521981 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.521973    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-sysctl-conf\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.522123 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.522014    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-cnibin\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.522188 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.522169    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06388ff-e2be-4364-ba0f-2d0b942e2243-host-var-lib-kubelet\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.522378 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.522361    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9e679c-eea6-4f88-9a5a-0f020d04f587-device-dir\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.525037 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.523012    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51e3d0e3-d802-42cb-89e8-29aed5331d2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.525935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.525435    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-tmp\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.525935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.525835    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e867e8f-9177-4f92-a6c7-e64255669905-agent-certs\") pod \"konnectivity-agent-hf98m\" (UID: \"6e867e8f-9177-4f92-a6c7-e64255669905\") " pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.527318 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.527287    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-etc-tuned\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.530019 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.529980    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6xk\" (UniqueName: \"kubernetes.io/projected/e90b820b-55c3-4ddd-bdbc-c5f9b02000d9-kube-api-access-ht6xk\") pod \"tuned-6dhv9\" (UID: \"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9\") " pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.530606 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.530587    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:40.530694 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.530612    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:40.530694 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.530626    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:40.530809 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:40.530703    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.030686302 +0000 UTC m=+3.434082374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:40.531983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.531941    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pjmv\" (UniqueName: \"kubernetes.io/projected/a06388ff-e2be-4364-ba0f-2d0b942e2243-kube-api-access-8pjmv\") pod \"multus-mql6s\" (UID: \"a06388ff-e2be-4364-ba0f-2d0b942e2243\") " pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.533039 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.532921    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2867\" (UniqueName: \"kubernetes.io/projected/704d581b-123b-42b3-a7d7-176251a58865-kube-api-access-z2867\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:40.533039 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.532948    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgpf\" (UniqueName: \"kubernetes.io/projected/914c2087-5c4e-4e81-a3eb-f573745db5b4-kube-api-access-gkgpf\") pod \"node-ca-kxqwg\" (UID: \"914c2087-5c4e-4e81-a3eb-f573745db5b4\") " pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.533213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.533184    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdcs\" (UniqueName: \"kubernetes.io/projected/51e3d0e3-d802-42cb-89e8-29aed5331d2f-kube-api-access-mxdcs\") pod \"multus-additional-cni-plugins-dpx2n\" (UID: \"51e3d0e3-d802-42cb-89e8-29aed5331d2f\") " pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.534708 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.534687    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5txm\" (UniqueName: \"kubernetes.io/projected/ea9e679c-eea6-4f88-9a5a-0f020d04f587-kube-api-access-m5txm\") pod \"aws-ebs-csi-driver-node-pwtnh\" (UID: \"ea9e679c-eea6-4f88-9a5a-0f020d04f587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.556999 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.556935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal" event={"ID":"ed96054e679b6790a225dcd3f67a3fd9","Type":"ContainerStarted","Data":"3dacd67ac1acf84db5168a998df8a3f6d4b7804388cdb10bb7a708b7ebf8af65"}
Jul 06 23:56:40.558305 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.558251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal" event={"ID":"200bd357d82b41d3210c766efd1f8f3e","Type":"ContainerStarted","Data":"590b1b37819781cdcf27bbdb06c4ae54b2504d0aee1b7323d6270c048bced9e3"}
Jul 06 23:56:40.622628 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b670d367-749c-4759-863b-02bd013fc4cf-iptables-alerter-script\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.622795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8slvz\" (UniqueName: \"kubernetes.io/projected/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-kube-api-access-8slvz\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.622795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-systemd-units\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.622795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-slash\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.622990 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-node-log\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.622990 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-log-socket\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.622990 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622964    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-slash\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623143 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.622999    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-node-log\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623143 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs57w\" (UniqueName: \"kubernetes.io/projected/b670d367-749c-4759-863b-02bd013fc4cf-kube-api-access-rs57w\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.623143 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-systemd-units\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623143 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623082    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-log-socket\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623143 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-netns\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-etc-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623208    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-etc-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623208    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-netns\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-kubelet\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjcx\" (UniqueName: \"kubernetes.io/projected/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-kube-api-access-lsjcx\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623274    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-kubelet\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-systemd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623305    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b670d367-749c-4759-863b-02bd013fc4cf-iptables-alerter-script\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.623375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623336    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-systemd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-script-lib\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-tmp-dir\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623525    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-netd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623570    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-config\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-env-overrides\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovn-node-metrics-cert\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-bin\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-var-lib-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623676    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-ovn\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b670d367-749c-4759-863b-02bd013fc4cf-host-slash\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-hosts-file\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.623794 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623792    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-run-ovn-kubernetes\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623829    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-netd\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623847    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-tmp-dir\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623922    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b670d367-749c-4759-863b-02bd013fc4cf-host-slash\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-var-lib-openvswitch\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.623981    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-hosts-file\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.624013    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-host-cni-bin\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.624047    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-run-ovn\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.624085    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-script-lib\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.624367    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovnkube-config\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.624561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.624378    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-env-overrides\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.626466 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.626445    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-ovn-node-metrics-cert\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.633003 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.632979    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjcx\" (UniqueName: \"kubernetes.io/projected/ffdb3454-b67e-4d5c-a9ae-9fd1992d9341-kube-api-access-lsjcx\") pod \"ovnkube-node-slnr9\" (UID: \"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341\") " pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.633262 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.633244    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs57w\" (UniqueName: \"kubernetes.io/projected/b670d367-749c-4759-863b-02bd013fc4cf-kube-api-access-rs57w\") pod \"iptables-alerter-xfjnl\" (UID: \"b670d367-749c-4759-863b-02bd013fc4cf\") " pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.633370 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.633355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8slvz\" (UniqueName: \"kubernetes.io/projected/a1ac883a-77ea-4b65-bc6f-9b184e9a5be0-kube-api-access-8slvz\") pod \"node-resolver-ls9cb\" (UID: \"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0\") " pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:40.707209 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.707171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mql6s"
Jul 06 23:56:40.716047 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.716025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh"
Jul 06 23:56:40.726545 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.726519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6dhv9"
Jul 06 23:56:40.732795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.732774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxqwg"
Jul 06 23:56:40.740239 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.740215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dpx2n"
Jul 06 23:56:40.748718 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.748698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:56:40.757228 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.757174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:56:40.766631 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.766612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xfjnl"
Jul 06 23:56:40.772067 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:40.772050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ls9cb"
Jul 06 23:56:41.026984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.026913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:41.026984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.026974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:41.027199 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.027079    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:41.027199 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.027140    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:42.027120064 +0000 UTC m=+4.430516144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:41.027199 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.027080    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:41.027350 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.027224    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:42.027211283 +0000 UTC m=+4.430607340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:41.127836 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.127806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:41.128013 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.127976    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:41.128013 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.127999    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:41.128013 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.128013    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:41.128145 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.128079    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:42.128060514 +0000 UTC m=+4.531456583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:41.452881 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.452782    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:40 +0000 UTC" deadline="2028-03-26 11:12:57.948144822 +0000 UTC"
Jul 06 23:56:41.452881 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.452815    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15083h16m16.495334008s"
Jul 06 23:56:41.538989 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.538768    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e867e8f_9177_4f92_a6c7_e64255669905.slice/crio-59cb934def821ceb5d829774e4b0d62aafc808813e3cd76db1e8208756b3dd0e WatchSource:0}: Error finding container 59cb934def821ceb5d829774e4b0d62aafc808813e3cd76db1e8208756b3dd0e: Status 404 returned error can't find the container with id 59cb934def821ceb5d829774e4b0d62aafc808813e3cd76db1e8208756b3dd0e
Jul 06 23:56:41.540665 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.540637    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffdb3454_b67e_4d5c_a9ae_9fd1992d9341.slice/crio-a2e3709664f010b378a4a0c45879cfd0a6a034a651e275ddba1eb00436b92bbe WatchSource:0}: Error finding container a2e3709664f010b378a4a0c45879cfd0a6a034a651e275ddba1eb00436b92bbe: Status 404 returned error can't find the container with id a2e3709664f010b378a4a0c45879cfd0a6a034a651e275ddba1eb00436b92bbe
Jul 06 23:56:41.543726 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.543707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06388ff_e2be_4364_ba0f_2d0b942e2243.slice/crio-b6f8df9b44490b10876a441164a1b7c61d7053ec8e1f20cb987ead95f112820e WatchSource:0}: Error finding container b6f8df9b44490b10876a441164a1b7c61d7053ec8e1f20cb987ead95f112820e: Status 404 returned error can't find the container with id b6f8df9b44490b10876a441164a1b7c61d7053ec8e1f20cb987ead95f112820e
Jul 06 23:56:41.545073 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.545036    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e3d0e3_d802_42cb_89e8_29aed5331d2f.slice/crio-388423d486cc0ad73ff944d636f68c3b789ebedc0d7ae3bdde23752298399a89 WatchSource:0}: Error finding container 388423d486cc0ad73ff944d636f68c3b789ebedc0d7ae3bdde23752298399a89: Status 404 returned error can't find the container with id 388423d486cc0ad73ff944d636f68c3b789ebedc0d7ae3bdde23752298399a89
Jul 06 23:56:41.548104 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.548082    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ac883a_77ea_4b65_bc6f_9b184e9a5be0.slice/crio-41ad9b27b6f140418bc1f4a2bda19421faba408519af0f8301ee39bbfaa5389b WatchSource:0}: Error finding container 41ad9b27b6f140418bc1f4a2bda19421faba408519af0f8301ee39bbfaa5389b: Status 404 returned error can't find the container with id 41ad9b27b6f140418bc1f4a2bda19421faba408519af0f8301ee39bbfaa5389b
Jul 06 23:56:41.553422 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:56:41.550685    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode90b820b_55c3_4ddd_bdbc_c5f9b02000d9.slice/crio-9547abe45f9bde3dc3a37b4dbcef91b0b9efd322a21561e5cba4e04caae2992f WatchSource:0}: Error finding container 9547abe45f9bde3dc3a37b4dbcef91b0b9efd322a21561e5cba4e04caae2992f: Status 404 returned error can't find the container with id 9547abe45f9bde3dc3a37b4dbcef91b0b9efd322a21561e5cba4e04caae2992f
Jul 06 23:56:41.553422 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.552181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:41.553422 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:41.552286    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:41.560760 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.560673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxqwg" event={"ID":"914c2087-5c4e-4e81-a3eb-f573745db5b4","Type":"ContainerStarted","Data":"152e1fff58e5eb7dbafa1e17c0efd4d6353b1d48f99c54a1e6bfce0960cb8ce5"}
Jul 06 23:56:41.562048 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.562023    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6dhv9" event={"ID":"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9","Type":"ContainerStarted","Data":"9547abe45f9bde3dc3a37b4dbcef91b0b9efd322a21561e5cba4e04caae2992f"}
Jul 06 23:56:41.563389 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.563367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ls9cb" event={"ID":"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0","Type":"ContainerStarted","Data":"41ad9b27b6f140418bc1f4a2bda19421faba408519af0f8301ee39bbfaa5389b"}
Jul 06 23:56:41.564690 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.564659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh" event={"ID":"ea9e679c-eea6-4f88-9a5a-0f020d04f587","Type":"ContainerStarted","Data":"4df3ffef63855c2eca99be2f47fdea14fc57c3a94e7231a0924a5d0b7dc9aa58"}
Jul 06 23:56:41.566015 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.565997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerStarted","Data":"388423d486cc0ad73ff944d636f68c3b789ebedc0d7ae3bdde23752298399a89"}
Jul 06 23:56:41.567183 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.567158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mql6s" event={"ID":"a06388ff-e2be-4364-ba0f-2d0b942e2243","Type":"ContainerStarted","Data":"b6f8df9b44490b10876a441164a1b7c61d7053ec8e1f20cb987ead95f112820e"}
Jul 06 23:56:41.568387 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.568368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xfjnl" event={"ID":"b670d367-749c-4759-863b-02bd013fc4cf","Type":"ContainerStarted","Data":"13ed99fce6e52f57ad6c5498c05236bd9ed64f582ad7163305d1aa693732c0c1"}
Jul 06 23:56:41.570115 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.570094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"a2e3709664f010b378a4a0c45879cfd0a6a034a651e275ddba1eb00436b92bbe"}
Jul 06 23:56:41.571324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:41.571306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hf98m" event={"ID":"6e867e8f-9177-4f92-a6c7-e64255669905","Type":"ContainerStarted","Data":"59cb934def821ceb5d829774e4b0d62aafc808813e3cd76db1e8208756b3dd0e"}
Jul 06 23:56:42.033577 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.033502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:42.033701 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.033578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:42.033755 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.033701    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:42.033812 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.033757    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:44.033740473 +0000 UTC m=+6.437136531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:42.034161 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.034141    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:42.034263 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.034196    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:44.034181899 +0000 UTC m=+6.437577955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:42.135659 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.135035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:42.135659 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.135251    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:42.135659 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.135276    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:42.135659 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.135291    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:42.135659 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.135349    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:44.135331873 +0000 UTC m=+6.538727931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:42.453612 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.453543    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:40 +0000 UTC" deadline="2028-03-15 01:49:49.186122658 +0000 UTC"
Jul 06 23:56:42.453612 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.453609    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14809h53m6.732518042s"
Jul 06 23:56:42.555781 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.555066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:42.555781 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.555194    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:42.555781 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.555620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:42.555781 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:42.555719    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:42.579372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.579293    2581 generic.go:356] "Generic (PLEG): container finished" podID="ed96054e679b6790a225dcd3f67a3fd9" containerID="9e280e8695d8977fc428a5a2a821d122e8f1fda37f0961acdf9caf1cef01b57f" exitCode=0
Jul 06 23:56:42.579536 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.579374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal" event={"ID":"ed96054e679b6790a225dcd3f67a3fd9","Type":"ContainerDied","Data":"9e280e8695d8977fc428a5a2a821d122e8f1fda37f0961acdf9caf1cef01b57f"}
Jul 06 23:56:42.595323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.594006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal" event={"ID":"200bd357d82b41d3210c766efd1f8f3e","Type":"ContainerStarted","Data":"5ab6e19e2b6de2867c0c9f99a2cad596c31b1cb0f69cbe329d3ee75a564bfcae"}
Jul 06 23:56:42.610297 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:42.609845    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-135.ec2.internal" podStartSLOduration=3.609823456 podStartE2EDuration="3.609823456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:42.609488773 +0000 UTC m=+5.012884856" watchObservedRunningTime="2026-07-06 23:56:42.609823456 +0000 UTC m=+5.013219532"
Jul 06 23:56:43.552439 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:43.552290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:43.552439 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:43.552419    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:43.624989 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:43.624921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal" event={"ID":"ed96054e679b6790a225dcd3f67a3fd9","Type":"ContainerStarted","Data":"df5c5efdfb9130aeabe4c20ecee02a4a2d4192ce9a72a50750fae440f0ad4c89"}
Jul 06 23:56:43.646055 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:43.644886    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-135.ec2.internal" podStartSLOduration=4.644866276 podStartE2EDuration="4.644866276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:43.644801442 +0000 UTC m=+6.048197519" watchObservedRunningTime="2026-07-06 23:56:43.644866276 +0000 UTC m=+6.048262373"
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:44.055900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:44.055985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.056124    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.056186    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.056167732 +0000 UTC m=+10.459563786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.056574    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:44.056719 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.056625    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.056610888 +0000 UTC m=+10.460006940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:44.157294 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:44.156640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:44.157294 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.156871    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:44.157294 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.156891    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:44.157294 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.156903    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:44.157294 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.156961    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.156942946 +0000 UTC m=+10.560339021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:44.555208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:44.555122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:44.555653 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.555253    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:44.555653 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:44.555625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:44.555775 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:44.555710    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:45.552033 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:45.552002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:45.552224 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:45.552121    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:46.553317 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:46.551919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:46.553317 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:46.551924    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:46.553317 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:46.552082    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:46.553317 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:46.552225    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:47.551743 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:47.551707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:47.551961 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:47.551844    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:48.091738 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:48.091699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:48.092201 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:48.091791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:48.092201 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.091925    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:48.092201 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.091992    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:56.091973201 +0000 UTC m=+18.495369257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:48.092201 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.091924    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:48.092201 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.092125    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:56.092101188 +0000 UTC m=+18.495497254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:48.193527 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:48.192991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:48.193527 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.193167    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:48.193527 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.193189    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:48.193527 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.193201    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:48.193527 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.193253    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:56.193236249 +0000 UTC m=+18.596632318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:48.553425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:48.552880    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:48.553425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:48.552892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:48.553425 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.553018    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:48.553425 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:48.553096    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:49.552258 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:49.552222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:49.552638 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:49.552354    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:50.552898 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:50.552469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:50.552898 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:50.552612    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:50.552898 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:50.552467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:50.552898 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:50.552825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:51.552650 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:51.552617    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:51.552803 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:51.552729    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:52.551926 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:52.551886    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:52.551926 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:52.551925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:52.552433 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:52.552028    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:52.552433 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:52.552143    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:53.552436 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:53.552395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:53.552900 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:53.552536    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:54.551811 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:54.551778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:54.551811 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:54.551822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:54.552074 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:54.551949    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:54.552122 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:54.552095    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:55.551736 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:55.551695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:55.552170 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:55.551836    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:56.151730 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:56.151698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:56.151935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:56.151765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:56.151935 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.151876    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:56.152060 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.151941    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.15192439 +0000 UTC m=+34.555320442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:56.152060 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.151876    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:56.152060 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.152014    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.152000902 +0000 UTC m=+34.555396958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:56.252140 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:56.252100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:56.252288 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.252272    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:56.252324 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.252294    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:56.252324 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.252303    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:56.252391 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.252356    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.25234338 +0000 UTC m=+34.655739447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:56.552474 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:56.552397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:56.552927 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:56.552397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:56.552927 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.552533    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:56.552927 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:56.552567    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:57.552034 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:57.552000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:57.552247 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:57.552146    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:56:58.555633 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:58.555604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:56:58.556094 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:58.555726    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:56:58.556094 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:58.555787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:56:58.556094 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:58.555896    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:56:59.552721 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:56:59.552660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:56:59.552908 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:56:59.552800    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:00.552207 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.552000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:00.552501 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:00.552328    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:00.552767 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.552749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:00.552882 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:00.552845    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:00.661426 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.661382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6dhv9" event={"ID":"e90b820b-55c3-4ddd-bdbc-c5f9b02000d9","Type":"ContainerStarted","Data":"eaa2e2f549e8a4ebed50e718ffca8e46b010923e7f5a3c257d4c48a844a81fe6"}
Jul 06 23:57:00.666489 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.666450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerStarted","Data":"bc72bbf014129de6d6b7c73882991f46ef859fc0045aaa4f6a4b923870757f0e"}
Jul 06 23:57:00.668251 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.668202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mql6s" event={"ID":"a06388ff-e2be-4364-ba0f-2d0b942e2243","Type":"ContainerStarted","Data":"82313f0db17ac1173e45e975214d89499cac561b3f33dd820569ad56d2f540c3"}
Jul 06 23:57:00.669624 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.669554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"a888838e291d08791e3cdafceb6641b4512e80d4417a1dbf6934c279efd6c984"}
Jul 06 23:57:00.671705 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.671634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hf98m" event={"ID":"6e867e8f-9177-4f92-a6c7-e64255669905","Type":"ContainerStarted","Data":"02e0e5db582b59b9207750fb6fa0c0698a1a274a70e31c7912d8b52a8f35e65b"}
Jul 06 23:57:00.682488 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.682427    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6dhv9" podStartSLOduration=3.762811329 podStartE2EDuration="22.68240484s" totalImagesPullingTime="18.919593511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:00.681119206 +0000 UTC m=+23.084515291" watchObservedRunningTime="2026-07-06 23:57:00.68240484 +0000 UTC m=+23.085800913"
Jul 06 23:57:00.701655 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.701593    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hf98m" podStartSLOduration=12.134779281 podStartE2EDuration="22.70157201s" totalImagesPullingTime="10.566792729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:00.700709903 +0000 UTC m=+23.104106009" watchObservedRunningTime="2026-07-06 23:57:00.70157201 +0000 UTC m=+23.104968083"
Jul 06 23:57:00.740304 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:00.740257    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mql6s" podStartSLOduration=3.761204425 podStartE2EDuration="22.740239498s" totalImagesPullingTime="18.979035073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:00.739801891 +0000 UTC m=+23.143197964" watchObservedRunningTime="2026-07-06 23:57:00.740239498 +0000 UTC m=+23.143635570"
Jul 06 23:57:01.552614 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.552435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:01.553278 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:01.552685    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:01.676176 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.675939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxqwg" event={"ID":"914c2087-5c4e-4e81-a3eb-f573745db5b4","Type":"ContainerStarted","Data":"5e6b9925fba1ee3363aa07ebd4cd6126d57241c8cd10d1cd86b001d197ee42cf"}
Jul 06 23:57:01.677586 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.677537    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ls9cb" event={"ID":"a1ac883a-77ea-4b65-bc6f-9b184e9a5be0","Type":"ContainerStarted","Data":"c4e6ef3c298fe4f2ab3bfaff76a82a4555aa601fb00545d4792848b10e8cf4b7"}
Jul 06 23:57:01.679105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.679082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh" event={"ID":"ea9e679c-eea6-4f88-9a5a-0f020d04f587","Type":"ContainerStarted","Data":"9444c4db6af84a5217234ed7b86248caa8436121d64f0876783b0c172f0c9689"}
Jul 06 23:57:01.680813 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.680791    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="bc72bbf014129de6d6b7c73882991f46ef859fc0045aaa4f6a4b923870757f0e" exitCode=0
Jul 06 23:57:01.680962 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.680892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"bc72bbf014129de6d6b7c73882991f46ef859fc0045aaa4f6a4b923870757f0e"}
Jul 06 23:57:01.684735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.684700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"787c72f0daa09a6e5d8dd8672aaff57da038c84308e9f575537d00c6b4ac316e"}
Jul 06 23:57:01.684845 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.684759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"e366810058aeb47682cc82333b7418abacadb03096b70bebf9e82e98bc03b4a8"}
Jul 06 23:57:01.684845 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.684795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"84bf1fdf376b4b34bc81f7622c32ead9ef6cc079163ff6551f7efe133cc28b99"}
Jul 06 23:57:01.684845 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.684826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"21dcb5d8359dd23101d29012a34c4f49b584fc71e8c993af0ffe8e10247a396b"}
Jul 06 23:57:01.685020 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.684885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"b6eaead01ad256be5ab989f25e09ae7a1f7d3a866a7397fd54af1443303ae00c"}
Jul 06 23:57:01.708282 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.708233    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kxqwg" podStartSLOduration=4.7910022770000005 podStartE2EDuration="23.708216938s" totalImagesPullingTime="18.917214661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:01.692985047 +0000 UTC m=+24.096381158" watchObservedRunningTime="2026-07-06 23:57:01.708216938 +0000 UTC m=+24.111613010"
Jul 06 23:57:01.708749 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:01.708719    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ls9cb" podStartSLOduration=4.789671197 podStartE2EDuration="23.708710518s" totalImagesPullingTime="18.919039321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:01.708199559 +0000 UTC m=+24.111595645" watchObservedRunningTime="2026-07-06 23:57:01.708710518 +0000 UTC m=+24.112106590"
Jul 06 23:57:02.183717 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.183680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:02.229542 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.229281    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 23:57:02.285621 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.285582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:02.388984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.388880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:02.429964 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.429934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:57:02.430625 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.430608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:57:02.462669 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.462640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:02.478809 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.478694    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:57:02.229536407Z","UUID":"e7ae9bce-c8cf-408f-8c8c-49b86bd06b12","Handler":null,"Name":"","Endpoint":""}
Jul 06 23:57:02.480169 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.480149    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 23:57:02.480278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.480192    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 23:57:02.552559 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.552520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:02.552744 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.552520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:02.552744 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:02.552640    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:02.553142 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:02.552755    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:02.675416 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.675334    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:02.688983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.688948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh" event={"ID":"ea9e679c-eea6-4f88-9a5a-0f020d04f587","Type":"ContainerStarted","Data":"7b5b9bf4b085b8249fcdf5c3b605343d008fb934265bdc2f3d30d0ae70f7dd1a"}
Jul 06 23:57:02.690478 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.690380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xfjnl" event={"ID":"b670d367-749c-4759-863b-02bd013fc4cf","Type":"ContainerStarted","Data":"98748306c90cc662bdb023d2d9859b45777344bf155edb1ee78abd276e60b686"}
Jul 06 23:57:02.690822 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.690796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:57:02.691128 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.691106    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hf98m"
Jul 06 23:57:02.850186 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:02.850123    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xfjnl" podStartSLOduration=5.933080785 podStartE2EDuration="24.85010066s" totalImagesPullingTime="18.917019875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:02.731895972 +0000 UTC m=+25.135292074" watchObservedRunningTime="2026-07-06 23:57:02.85010066 +0000 UTC m=+25.253496737"
Jul 06 23:57:03.091186 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.091151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:03.269331 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.269296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:03.551801 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.551776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:03.551951 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:03.551913    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:03.697096 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.697020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"97fe6dcf14604d9733d8a6c45fb29726a449898b35f482818246fcd5d1b9aa45"}
Jul 06 23:57:03.699291 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.699252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh" event={"ID":"ea9e679c-eea6-4f88-9a5a-0f020d04f587","Type":"ContainerStarted","Data":"3a02cb27bc28d1523bbb8625aebdfc324c18c57e493cb396a7185ebbcb429c7c"}
Jul 06 23:57:03.722188 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:03.722128    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwtnh" podStartSLOduration=4.197057919 podStartE2EDuration="25.722107261s" totalImagesPullingTime="21.525049342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:03.721792228 +0000 UTC m=+26.125188302" watchObservedRunningTime="2026-07-06 23:57:03.722107261 +0000 UTC m=+26.125503335"
Jul 06 23:57:04.258872 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:04.258837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:04.552611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:04.552538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:04.552748 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:04.552663    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:04.552748 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:04.552717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:04.552868 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:04.552832    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:04.667083 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:04.667047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:04.856674 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:04.856591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:05.059823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:05.059790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:05.260425 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:05.260368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:05.552373 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:05.552288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:05.552538 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:05.552421    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:05.853892 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:05.853721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:05.984040 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:05.984011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:06.084485 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.084453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:06.462055 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.461969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:06.552455 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.552426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:06.552455 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.552445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:06.552654 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:06.552534    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:06.552691 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:06.552666    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:06.707709 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.707677    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="153e435486ec4700aa809847b32144197c9288b98dad7352acfdf79ee4662b81" exitCode=0
Jul 06 23:57:06.707898 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.707762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"153e435486ec4700aa809847b32144197c9288b98dad7352acfdf79ee4662b81"}
Jul 06 23:57:06.711436 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.711410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" event={"ID":"ffdb3454-b67e-4d5c-a9ae-9fd1992d9341","Type":"ContainerStarted","Data":"07492e9de438d7ea04668c0748fc106bd28c9120b9a0e44cc62f5213365bfc2f"}
Jul 06 23:57:06.711784 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.711763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:06.711906 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.711847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:06.711906 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.711895    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:06.727108 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.727084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:06.727265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.727219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:06.754108 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.754061    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9" podStartSLOduration=9.763265204 podStartE2EDuration="28.754046784s" totalImagesPullingTime="18.99078158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:06.7540251 +0000 UTC m=+29.157421189" watchObservedRunningTime="2026-07-06 23:57:06.754046784 +0000 UTC m=+29.157442854"
Jul 06 23:57:06.859910 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.859872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:06.926547 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:06.926514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:07.259073 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:07.259038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:07.453181 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:07.453154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:07.552135 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:07.552059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:07.552293 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:07.552190    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:07.861141 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:07.861106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:08.261757 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.261724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:08.376990 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.376958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:08.553212 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.553130    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:08.553356 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:08.553222    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:08.553356 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.553334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:08.553450 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:08.553432    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:08.716309 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.716272    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="a64eaca9810c124286e03e49d07b5bbc188f8e7944429b44d873fb5b506805c8" exitCode=0
Jul 06 23:57:08.716457 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:08.716353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"a64eaca9810c124286e03e49d07b5bbc188f8e7944429b44d873fb5b506805c8"}
Jul 06 23:57:09.055546 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:09.055507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:09.450954 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:09.450877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:09.552402 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:09.552372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:09.552546 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:09.552468    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:09.654603 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:09.654569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:09.854283 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:09.854258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:10.268953 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.268918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:10.552662 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.552593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:10.553018 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:10.552684    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:10.553018 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.552732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:10.553018 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:10.552788    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:10.722707 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.722671    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="18c6d90458c1ab6061cf07090c3920db440dba389655856fa6422ebda511588a" exitCode=0
Jul 06 23:57:10.722908 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.722739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"18c6d90458c1ab6061cf07090c3920db440dba389655856fa6422ebda511588a"}
Jul 06 23:57:10.858365 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:10.858276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.040467 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.040428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.141390 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.141287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.242490 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.242441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.299132 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.299088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.552397 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.552362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:11.552577 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:11.552480    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:11.655179 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.655145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:11.853585 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:11.853503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:12.166825 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.166737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:12.166825 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.166795    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:12.167055 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.166913    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:12.167055 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.166975    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs podName:704d581b-123b-42b3-a7d7-176251a58865 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.166956222 +0000 UTC m=+66.570352291 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs") pod "network-metrics-daemon-s7xxc" (UID: "704d581b-123b-42b3-a7d7-176251a58865") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:12.167055 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.166915    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:12.167055 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.167052    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret podName:5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.167033309 +0000 UTC m=+66.570429361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret") pod "global-pull-secret-syncer-hp5lp" (UID: "5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:12.256665 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.256631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:12.267506 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.267480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:12.267647 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.267638    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:57:12.267713 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.267657    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:57:12.267713 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.267670    2581 projected.go:196] Error preparing data for projected volume kube-api-access-249np for pod openshift-network-diagnostics/network-check-target-6llz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:12.267806 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.267733    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np podName:4ca593e8-3b94-4801-b466-70d71346e978 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.26771609 +0000 UTC m=+66.671112143 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-249np" (UniqueName: "kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np") pod "network-check-target-6llz8" (UID: "4ca593e8-3b94-4801-b466-70d71346e978") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:12.552596 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.552562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:12.552763 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.552689    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:12.552763 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.552746    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:12.552939 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:12.552865    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:12.724583 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.724524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:12.881503 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:12.881415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:13.551822 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:13.551784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:13.552011 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:13.551915    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:14.552257 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:14.552225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:14.552708 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:14.552225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:14.552708 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:14.552342    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:14.552708 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:14.552460    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:15.552151 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:15.552119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:15.552310 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:15.552219    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:15.735661 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:15.735627    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="deb9fc5ab4954eeccab43b8f9a2ba53cef0e71548f7a42f21da9f6f7ef62a2ce" exitCode=0
Jul 06 23:57:15.735832 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:15.735676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"deb9fc5ab4954eeccab43b8f9a2ba53cef0e71548f7a42f21da9f6f7ef62a2ce"}
Jul 06 23:57:16.552201 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:16.552166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:16.552372 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:16.552263    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:16.552372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:16.552326    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:16.552658 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:16.552409    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:16.740594 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:16.740562    2581 generic.go:356] "Generic (PLEG): container finished" podID="51e3d0e3-d802-42cb-89e8-29aed5331d2f" containerID="97ee03c03ee94c00dfd26d36e67fa170279d91a7a6e36413d4b82a064ab5f5cf" exitCode=0
Jul 06 23:57:16.740734 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:16.740603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerDied","Data":"97ee03c03ee94c00dfd26d36e67fa170279d91a7a6e36413d4b82a064ab5f5cf"}
Jul 06 23:57:17.552468 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:17.552438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:17.552927 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:17.552526    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:17.745293 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:17.745246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" event={"ID":"51e3d0e3-d802-42cb-89e8-29aed5331d2f","Type":"ContainerStarted","Data":"dd4cdced5c4c3c6ac20e79657d0ee94ef50c58c49764b33c7858642471cfd7da"}
Jul 06 23:57:17.782007 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:17.781963    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dpx2n" podStartSLOduration=10.57886074 podStartE2EDuration="39.781948484s" totalImagesPullingTime="29.203087744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.780397519 +0000 UTC m=+40.183793593" watchObservedRunningTime="2026-07-06 23:57:17.781948484 +0000 UTC m=+40.185344556"
Jul 06 23:57:18.552751 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:18.552722    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:18.553120 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:18.552799    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:18.553120 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:18.552930    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:18.553120 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:18.553055    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:19.551891 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:19.551837    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:19.552059 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:19.551962    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:20.552213 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:20.552174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:20.552655 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:20.552311    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:20.552655 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:20.552353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:20.552655 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:20.552436    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:21.134209 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.134172    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:21.248923 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.248892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:21.362026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.361992    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:21.472185 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.472155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:21.552334 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.552309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:21.552666 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:21.552407    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:21.572816 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:21.572776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:22.012902 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.012845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:22.094606 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.094574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6llz8"]
Jul 06 23:57:22.094776 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.094698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:22.094836 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:22.094778    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:22.097431 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.097402    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hp5lp"]
Jul 06 23:57:22.097583 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.097538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:22.097668 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:22.097644    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:22.098113 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.098091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7xxc"]
Jul 06 23:57:22.098222 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.098194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:22.098383 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:22.098308    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:22.400763 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.400729    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:22.470884 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:22.470836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:23.551848 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:23.551819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:23.551848 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:23.551844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:23.552553 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:23.551926    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:23.552553 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:23.551963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:23.552553 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:23.552025    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:23.552553 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:23.552110    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:24.292307 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:24.292200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:24.458955 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:24.458924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:24.710106 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:24.709920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:25.551733 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:25.551693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:25.551733 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:25.551733    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:25.551971 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:25.551712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:25.551971 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:25.551815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s7xxc" podUID="704d581b-123b-42b3-a7d7-176251a58865"
Jul 06 23:57:25.551971 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:25.551876    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6llz8" podUID="4ca593e8-3b94-4801-b466-70d71346e978"
Jul 06 23:57:25.551971 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:25.551950    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hp5lp" podUID="5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63"
Jul 06 23:57:26.135299 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.135251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:26.234917 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.234767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:26.376919 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.376892    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-135.ec2.internal" event="NodeReady"
Jul 06 23:57:26.377101 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.377020    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 23:57:26.453608 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.453575    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7tsxx"]
Jul 06 23:57:26.478403 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.478382    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pvsp8"]
Jul 06 23:57:26.478540 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.478504    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.488296 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.488271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 23:57:26.488573 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.488555    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf85g\""
Jul 06 23:57:26.488770 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.488757    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 23:57:26.492459 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.492437    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7tsxx"]
Jul 06 23:57:26.492565 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.492468    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pvsp8"]
Jul 06 23:57:26.492565 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.492549    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.498745 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.498728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 23:57:26.499404 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.499384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 23:57:26.499509 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.499429    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 23:57:26.499509 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.499458    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q9bzd\""
Jul 06 23:57:26.499509 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.499470    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 23:57:26.554851 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.554825    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7j9lz"]
Jul 06 23:57:26.573039 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.573012    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.573039 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.573014    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7j9lz"]
Jul 06 23:57:26.577219 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edb12a13-576a-4e8c-9b51-4ecf16debbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.577360 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xbxm\" (UniqueName: \"kubernetes.io/projected/44018009-fed8-42aa-944b-3095c06fa355-kube-api-access-2xbxm\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.577360 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-metrics-tls\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.577360 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edb12a13-576a-4e8c-9b51-4ecf16debbf4-crio-socket\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.577494 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-config-volume\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.577494 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.577494 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-tmp-dir\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.577637 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edb12a13-576a-4e8c-9b51-4ecf16debbf4-data-volume\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.577637 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577586    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 23:57:26.577637 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jbt\" (UniqueName: \"kubernetes.io/projected/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-api-access-k4jbt\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.577823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577641    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpwtc\""
Jul 06 23:57:26.577823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvdz\" (UniqueName: \"kubernetes.io/projected/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-kube-api-access-4pvdz\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.577823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577678    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 23:57:26.577823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44018009-fed8-42aa-944b-3095c06fa355-cert\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.578045 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.577971    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 23:57:26.679179 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44018009-fed8-42aa-944b-3095c06fa355-cert\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.679372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edb12a13-576a-4e8c-9b51-4ecf16debbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.679372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xbxm\" (UniqueName: \"kubernetes.io/projected/44018009-fed8-42aa-944b-3095c06fa355-kube-api-access-2xbxm\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.679372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-metrics-tls\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.679372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edb12a13-576a-4e8c-9b51-4ecf16debbf4-crio-socket\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.679372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-config-volume\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.679598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.679598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-tmp-dir\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.679598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edb12a13-576a-4e8c-9b51-4ecf16debbf4-data-volume\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.679598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jbt\" (UniqueName: \"kubernetes.io/projected/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-api-access-k4jbt\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.679598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.679583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvdz\" (UniqueName: \"kubernetes.io/projected/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-kube-api-access-4pvdz\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.682368 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.682309    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-tmp-dir\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.682483 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.682442    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/edb12a13-576a-4e8c-9b51-4ecf16debbf4-crio-socket\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.682565 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.682543    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.682607 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.682581    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-config-volume\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.683374 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.683353    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/edb12a13-576a-4e8c-9b51-4ecf16debbf4-data-volume\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.684465 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.684445    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/edb12a13-576a-4e8c-9b51-4ecf16debbf4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.684539 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.684487    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-metrics-tls\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.684602 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.684584    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44018009-fed8-42aa-944b-3095c06fa355-cert\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.691210 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.691191    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvdz\" (UniqueName: \"kubernetes.io/projected/5a212e00-f6d3-41cc-8cf0-63bfa3dc9560-kube-api-access-4pvdz\") pod \"dns-default-7tsxx\" (UID: \"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560\") " pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.693335 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.693304    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xbxm\" (UniqueName: \"kubernetes.io/projected/44018009-fed8-42aa-944b-3095c06fa355-kube-api-access-2xbxm\") pod \"ingress-canary-7j9lz\" (UID: \"44018009-fed8-42aa-944b-3095c06fa355\") " pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.693871 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.693839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jbt\" (UniqueName: \"kubernetes.io/projected/edb12a13-576a-4e8c-9b51-4ecf16debbf4-kube-api-access-k4jbt\") pod \"insights-runtime-extractor-pvsp8\" (UID: \"edb12a13-576a-4e8c-9b51-4ecf16debbf4\") " pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.719573 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.719538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:26.786983 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.786906    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:26.787764 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.787740    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:26.800909 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.800890    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pvsp8"
Jul 06 23:57:26.881235 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.881198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7j9lz"
Jul 06 23:57:26.991831 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.991794    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pvsp8"]
Jul 06 23:57:26.994722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:26.994696    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7tsxx"]
Jul 06 23:57:26.995355 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:26.995332    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb12a13_576a_4e8c_9b51_4ecf16debbf4.slice/crio-187cc871ca94096c759661bfffc6ed08827e46d370537c5a95558582a6c949ef WatchSource:0}: Error finding container 187cc871ca94096c759661bfffc6ed08827e46d370537c5a95558582a6c949ef: Status 404 returned error can't find the container with id 187cc871ca94096c759661bfffc6ed08827e46d370537c5a95558582a6c949ef
Jul 06 23:57:26.997609 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:26.997587    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a212e00_f6d3_41cc_8cf0_63bfa3dc9560.slice/crio-2d90a6ad083820b35c98c4695791b611388ab94e8d5b8bad6cb139eb3d21f5ab WatchSource:0}: Error finding container 2d90a6ad083820b35c98c4695791b611388ab94e8d5b8bad6cb139eb3d21f5ab: Status 404 returned error can't find the container with id 2d90a6ad083820b35c98c4695791b611388ab94e8d5b8bad6cb139eb3d21f5ab
Jul 06 23:57:27.047265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.047188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7j9lz"]
Jul 06 23:57:27.049969 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:27.049941    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44018009_fed8_42aa_944b_3095c06fa355.slice/crio-28b6cd4b6cb0683522d2cdbd1a812ebd4322afbc9dae85060d11991e4b9a3746 WatchSource:0}: Error finding container 28b6cd4b6cb0683522d2cdbd1a812ebd4322afbc9dae85060d11991e4b9a3746: Status 404 returned error can't find the container with id 28b6cd4b6cb0683522d2cdbd1a812ebd4322afbc9dae85060d11991e4b9a3746
Jul 06 23:57:27.552717 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.552679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:27.553171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.552802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:27.553171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.552760    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:27.557273 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.556971    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:57:27.557273 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.557045    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:57:27.557273 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.557080    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:57:27.557273 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.556971    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-knf8z\""
Jul 06 23:57:27.557273 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.557183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lswch\""
Jul 06 23:57:27.557615 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.557392    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:57:27.769805 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.769557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7j9lz" event={"ID":"44018009-fed8-42aa-944b-3095c06fa355","Type":"ContainerStarted","Data":"28b6cd4b6cb0683522d2cdbd1a812ebd4322afbc9dae85060d11991e4b9a3746"}
Jul 06 23:57:27.771317 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.771264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pvsp8" event={"ID":"edb12a13-576a-4e8c-9b51-4ecf16debbf4","Type":"ContainerStarted","Data":"187cc871ca94096c759661bfffc6ed08827e46d370537c5a95558582a6c949ef"}
Jul 06 23:57:27.772627 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:27.772603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tsxx" event={"ID":"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560","Type":"ContainerStarted","Data":"2d90a6ad083820b35c98c4695791b611388ab94e8d5b8bad6cb139eb3d21f5ab"}
Jul 06 23:57:28.515401 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.515365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:28.559730 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.559699    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"]
Jul 06 23:57:28.571186 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.571158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.575995 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.575961    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 23:57:28.576396 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.576376    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 06 23:57:28.577081 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.577060    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 06 23:57:28.577182 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.577123    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-58wsz\""
Jul 06 23:57:28.577775 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.577754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 23:57:28.577875 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.577832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 23:57:28.579135 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.579115    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"]
Jul 06 23:57:28.595097 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.595060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86651ac1-28cc-4309-bc1b-436402a8fa72-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.595282 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.595160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.595282 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.595199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgh74\" (UniqueName: \"kubernetes.io/projected/86651ac1-28cc-4309-bc1b-436402a8fa72-kube-api-access-zgh74\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.595282 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.595237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.600137 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.600109    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:28.695625 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.695588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.695773 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.695631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgh74\" (UniqueName: \"kubernetes.io/projected/86651ac1-28cc-4309-bc1b-436402a8fa72-kube-api-access-zgh74\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.695773 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.695678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.695773 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.695742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86651ac1-28cc-4309-bc1b-436402a8fa72-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.696090 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:28.696070    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 06 23:57:28.696209 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:28.696137    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls podName:86651ac1-28cc-4309-bc1b-436402a8fa72 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:29.196116238 +0000 UTC m=+51.599512289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-8lspz" (UID: "86651ac1-28cc-4309-bc1b-436402a8fa72") : secret "prometheus-operator-tls" not found
Jul 06 23:57:28.696604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.696553    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86651ac1-28cc-4309-bc1b-436402a8fa72-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.700233 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.700209    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.711401 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.711359    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgh74\" (UniqueName: \"kubernetes.io/projected/86651ac1-28cc-4309-bc1b-436402a8fa72-kube-api-access-zgh74\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:28.830263 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.830177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:28.896592 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:28.896561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:29.162184 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:29.162100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:29.199757 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:29.199717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:29.202704 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:29.202672    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86651ac1-28cc-4309-bc1b-436402a8fa72-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8lspz\" (UID: \"86651ac1-28cc-4309-bc1b-436402a8fa72\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:29.484952 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:29.484916    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"
Jul 06 23:57:30.415449 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:30.415411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:31.147635 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:31.147605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:31.209010 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:31.208983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:31.723039 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:31.723004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:31.786344 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:31.786307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:33.094310 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.094117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz"]
Jul 06 23:57:33.126998 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:33.126969    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86651ac1_28cc_4309_bc1b_436402a8fa72.slice/crio-54132b45bc3418686a6c8f56901baa53eb9dc496a1965ac163bdf70ca9ec94e8 WatchSource:0}: Error finding container 54132b45bc3418686a6c8f56901baa53eb9dc496a1965ac163bdf70ca9ec94e8: Status 404 returned error can't find the container with id 54132b45bc3418686a6c8f56901baa53eb9dc496a1965ac163bdf70ca9ec94e8
Jul 06 23:57:33.793287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.792978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7j9lz" event={"ID":"44018009-fed8-42aa-944b-3095c06fa355","Type":"ContainerStarted","Data":"79c47199e99049a24b0d8b6256176493d838cf446ba1fac7b02908c9315b4ffa"}
Jul 06 23:57:33.795169 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.795114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tsxx" event={"ID":"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560","Type":"ContainerStarted","Data":"6b36d4f46b4da24efeed7afa2771f754e1b96453620990583b6738411efde328"}
Jul 06 23:57:33.795313 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.795198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7tsxx" event={"ID":"5a212e00-f6d3-41cc-8cf0-63bfa3dc9560","Type":"ContainerStarted","Data":"dafcffad94a0e5ef50c14a5ad40deae8951dc1fbca4a7859664c9e9b229a3d81"}
Jul 06 23:57:33.795313 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.795230    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:33.796935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.796874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pvsp8" event={"ID":"edb12a13-576a-4e8c-9b51-4ecf16debbf4","Type":"ContainerStarted","Data":"7b53473e925c0e699701efec0de0491a9647daa6a52c274359669f547f4b8711"}
Jul 06 23:57:33.798265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.798236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz" event={"ID":"86651ac1-28cc-4309-bc1b-436402a8fa72","Type":"ContainerStarted","Data":"54132b45bc3418686a6c8f56901baa53eb9dc496a1965ac163bdf70ca9ec94e8"}
Jul 06 23:57:33.819302 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.819249    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7j9lz" podStartSLOduration=1.921303824 podStartE2EDuration="7.819229615s" totalImagesPullingTime="5.897925791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:33.818645567 +0000 UTC m=+56.222041677" watchObservedRunningTime="2026-07-06 23:57:33.819229615 +0000 UTC m=+56.222625690"
Jul 06 23:57:33.842459 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:33.839688    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7tsxx" podStartSLOduration=1.890154865 podStartE2EDuration="7.839664609s" totalImagesPullingTime="5.949509744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:33.837484425 +0000 UTC m=+56.240880500" watchObservedRunningTime="2026-07-06 23:57:33.839664609 +0000 UTC m=+56.243060682"
Jul 06 23:57:34.340304 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.340280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:34.443445 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.443410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:34.539422 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.539337    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:34.804426 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.804342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pvsp8" event={"ID":"edb12a13-576a-4e8c-9b51-4ecf16debbf4","Type":"ContainerStarted","Data":"5a669aa94818b41becc3167adfc70bcbadcbf47e4a56459c5be16e0f80dd4374"}
Jul 06 23:57:34.806395 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.806335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz" event={"ID":"86651ac1-28cc-4309-bc1b-436402a8fa72","Type":"ContainerStarted","Data":"c2245f577c0cf38c69dab33d68de2f448ae23102dea7addf71235e01a8430b2f"}
Jul 06 23:57:34.806395 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.806385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz" event={"ID":"86651ac1-28cc-4309-bc1b-436402a8fa72","Type":"ContainerStarted","Data":"0ef14a99aec8f9cad056f0737b20e6951d7fa66cc091ca75be6ccb9e914baa80"}
Jul 06 23:57:34.827323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:34.827258    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8lspz" podStartSLOduration=5.696421314 podStartE2EDuration="6.82724348s" totalImagesPullingTime="1.130822166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:34.825401697 +0000 UTC m=+57.228797790" watchObservedRunningTime="2026-07-06 23:57:34.82724348 +0000 UTC m=+57.230639552"
Jul 06 23:57:36.691498 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:36.691470    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:36.817291 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:36.817256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pvsp8" event={"ID":"edb12a13-576a-4e8c-9b51-4ecf16debbf4","Type":"ContainerStarted","Data":"24f3f861e1248d6b6f2da3c8cd5104c254d3abf0a5781408141c47103af2ddab"}
Jul 06 23:57:36.847926 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:36.847878    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pvsp8" podStartSLOduration=1.6445701339999999 podStartE2EDuration="10.847846982s" totalImagesPullingTime="9.203276848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:36.846550682 +0000 UTC m=+59.249946756" watchObservedRunningTime="2026-07-06 23:57:36.847846982 +0000 UTC m=+59.251243054"
Jul 06 23:57:36.970714 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:36.970683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.010969 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.010943    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"]
Jul 06 23:57:37.012944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.012926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.015345 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.015318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"]
Jul 06 23:57:37.015604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.015587    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-246gv\""
Jul 06 23:57:37.015690 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.015669    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 06 23:57:37.016019 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.015999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 06 23:57:37.017186 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.017169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.020607 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.020590    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 23:57:37.021184 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.021165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 23:57:37.021280 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.021171    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bp7qs\""
Jul 06 23:57:37.021280 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.021271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 23:57:37.026974 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.026955    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"]
Jul 06 23:57:37.031010 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.030992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"]
Jul 06 23:57:37.049288 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.049257    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z5gzj"]
Jul 06 23:57:37.051152 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.051136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.053744 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.053722    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 23:57:37.054028 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.054013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 23:57:37.054028 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.054020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 23:57:37.054248 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.054236    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7qp4\""
Jul 06 23:57:37.067824 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.067791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fdt\" (UniqueName: \"kubernetes.io/projected/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-api-access-q9fdt\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.067824 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.067832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.067979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.067850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.067979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.067886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.067979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.067927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.068071 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.068003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl5vh\" (UniqueName: \"kubernetes.io/projected/1773a6ee-bff4-404c-9bbc-ad641b078b9a-kube-api-access-zl5vh\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.068071 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.068035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.068071 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.068067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.068164 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.068087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1773a6ee-bff4-404c-9bbc-ad641b078b9a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.068164 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.068117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.089317 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.089286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.168721 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.168721 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.168939 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtl2\" (UniqueName: \"kubernetes.io/projected/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-kube-api-access-8jtl2\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.168939 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl5vh\" (UniqueName: \"kubernetes.io/projected/1773a6ee-bff4-404c-9bbc-ad641b078b9a-kube-api-access-zl5vh\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.168939 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-metrics-client-ca\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.168939 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.168939 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-root\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169134 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.168942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169134 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.169134 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-tls\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-textfile\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1773a6ee-bff4-404c-9bbc-ad641b078b9a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.169287 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.169505 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.169505 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-wtmp\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169505 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.169505 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fdt\" (UniqueName: \"kubernetes.io/projected/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-api-access-q9fdt\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.169663 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-sys\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.169663 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.170148 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.169897    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1773a6ee-bff4-404c-9bbc-ad641b078b9a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.170148 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.170107    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.170301 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.170126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.172724 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.172702    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.172823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.172750    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1773a6ee-bff4-404c-9bbc-ad641b078b9a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.172884 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.172817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.172925 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.172887    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.178970 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.178946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.180234 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.180204    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fdt\" (UniqueName: \"kubernetes.io/projected/ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1-kube-api-access-q9fdt\") pod \"kube-state-metrics-7c9dbb8b7d-stvj5\" (UID: \"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.180310 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.180248    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl5vh\" (UniqueName: \"kubernetes.io/projected/1773a6ee-bff4-404c-9bbc-ad641b078b9a-kube-api-access-zl5vh\") pod \"openshift-state-metrics-54cf96dfcf-4l7rc\" (UID: \"1773a6ee-bff4-404c-9bbc-ad641b078b9a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.270345 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-sys\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270345 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtl2\" (UniqueName: \"kubernetes.io/projected/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-kube-api-access-8jtl2\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270345 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-metrics-client-ca\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270560 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270396    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-sys\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270560 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-root\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270635 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270556    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-root\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270673 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270706 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-tls\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270706 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270786 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-textfile\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.270996 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-wtmp\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.271103 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.270994    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-textfile\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.271103 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.271048    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-wtmp\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.271103 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.271093    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-metrics-client-ca\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.271341 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.271248    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.273074 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.273054    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.273157 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.273060    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-node-exporter-tls\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.276170 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.276147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.278891 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.278871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtl2\" (UniqueName: \"kubernetes.io/projected/48b7d031-87ca-4fe0-b7ec-ae5554d05a7c-kube-api-access-8jtl2\") pod \"node-exporter-z5gzj\" (UID: \"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c\") " pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.323355 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.323324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"
Jul 06 23:57:37.329170 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.329149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"
Jul 06 23:57:37.359486 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.359451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z5gzj"
Jul 06 23:57:37.368664 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:37.368613    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b7d031_87ca_4fe0_b7ec_ae5554d05a7c.slice/crio-1b65e70ce848487742bccea9ff871a64420c59b6efa02f5393b7041990c61f02 WatchSource:0}: Error finding container 1b65e70ce848487742bccea9ff871a64420c59b6efa02f5393b7041990c61f02: Status 404 returned error can't find the container with id 1b65e70ce848487742bccea9ff871a64420c59b6efa02f5393b7041990c61f02
Jul 06 23:57:37.385416 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.382595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.469604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.469574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc"]
Jul 06 23:57:37.472624 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.472598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.472728 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:37.472642    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1773a6ee_bff4_404c_9bbc_ad641b078b9a.slice/crio-88a8ce08778000bdfff4028e74667b7c1a79d1320c628136c58040aa479cad83 WatchSource:0}: Error finding container 88a8ce08778000bdfff4028e74667b7c1a79d1320c628136c58040aa479cad83: Status 404 returned error can't find the container with id 88a8ce08778000bdfff4028e74667b7c1a79d1320c628136c58040aa479cad83
Jul 06 23:57:37.481553 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.481528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5"]
Jul 06 23:57:37.485055 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:37.485035    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5355cc_fbfe_45c6_b4d6_92e7a7c039f1.slice/crio-b85ee0b528d8fb5ee08bf3a8582bfe98af599fa8839a2674678e9b2a904fe784 WatchSource:0}: Error finding container b85ee0b528d8fb5ee08bf3a8582bfe98af599fa8839a2674678e9b2a904fe784: Status 404 returned error can't find the container with id b85ee0b528d8fb5ee08bf3a8582bfe98af599fa8839a2674678e9b2a904fe784
Jul 06 23:57:37.715540 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.715506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:37.822281 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.822188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5" event={"ID":"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1","Type":"ContainerStarted","Data":"b85ee0b528d8fb5ee08bf3a8582bfe98af599fa8839a2674678e9b2a904fe784"}
Jul 06 23:57:37.823519 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.823492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5gzj" event={"ID":"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c","Type":"ContainerStarted","Data":"1b65e70ce848487742bccea9ff871a64420c59b6efa02f5393b7041990c61f02"}
Jul 06 23:57:37.825612 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.825582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc" event={"ID":"1773a6ee-bff4-404c-9bbc-ad641b078b9a","Type":"ContainerStarted","Data":"172bb929b9663f5d729254a4ca197bc84f07c2f145eb5d8f0f884fd41b62193d"}
Jul 06 23:57:37.825721 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.825640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc" event={"ID":"1773a6ee-bff4-404c-9bbc-ad641b078b9a","Type":"ContainerStarted","Data":"ffcf881cbe605ff09c425f9675a3eecc9171904d83cfb00d89fada9d291e6e2b"}
Jul 06 23:57:37.825721 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.825670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc" event={"ID":"1773a6ee-bff4-404c-9bbc-ad641b078b9a","Type":"ContainerStarted","Data":"88a8ce08778000bdfff4028e74667b7c1a79d1320c628136c58040aa479cad83"}
Jul 06 23:57:37.891701 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:37.891660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.010219 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.010184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.110132 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.110009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.226045 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.226005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.229574 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.229545    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:38.236555 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.235709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.241560    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.241781    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242032    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242092    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242237    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242301    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242429    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znb9k\""
Jul 06 23:57:38.242579 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.242490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 23:57:38.243774 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.243746    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:57:38.244131 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.243965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 23:57:38.257057 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.257033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:38.279638 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.279785 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9l52\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.279785 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.279948 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280013 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280013 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.279996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280114 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280114 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280114 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.280406 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.280257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.381515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.382124 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9l52\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.382124 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.382124 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.381629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:38.383796    2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:57:38.383877    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls podName:7c3f6b14-64cb-48b2-a97a-c58bb7856906 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:38.883841572 +0000 UTC m=+61.287237624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906") : secret "alertmanager-main-tls" not found
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.384177    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.384422    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.384572    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385051 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.384737    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385625 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.385602    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385880 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.385823    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.385990 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.385968    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.386164 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.386088    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.386164 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.386138    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.387151 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.387126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.392383 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.392334    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:57:38.395601 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.395577    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.407828 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.407804    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9l52\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.631233 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.631162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.728936 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.728904    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slnr9"
Jul 06 23:57:38.886141 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.886094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.888462 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.888434    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:38.925605 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:38.925569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:39.028905 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.028879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:39.154131 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.154100    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znb9k\""
Jul 06 23:57:39.161707 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.161681    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:57:39.329093 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.329061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:57:39.332744 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:39.332719    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3f6b14_64cb_48b2_a97a_c58bb7856906.slice/crio-403dba610d2024384f00712dedc6cbba9648bfcd01724358bca1a521c44a1e4e WatchSource:0}: Error finding container 403dba610d2024384f00712dedc6cbba9648bfcd01724358bca1a521c44a1e4e: Status 404 returned error can't find the container with id 403dba610d2024384f00712dedc6cbba9648bfcd01724358bca1a521c44a1e4e
Jul 06 23:57:39.835758 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.835722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc" event={"ID":"1773a6ee-bff4-404c-9bbc-ad641b078b9a","Type":"ContainerStarted","Data":"02a113c8b196e82a132ab66484e46b102cc9ce5f761c8fff8cac22924b40d4b0"}
Jul 06 23:57:39.837616 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.837584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"403dba610d2024384f00712dedc6cbba9648bfcd01724358bca1a521c44a1e4e"}
Jul 06 23:57:39.840038 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.840014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5" event={"ID":"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1","Type":"ContainerStarted","Data":"8d8cb317fe085803b9d772e563eb7e52ad36679bf7d3a5846775653278a938ec"}
Jul 06 23:57:39.840216 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.840198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5" event={"ID":"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1","Type":"ContainerStarted","Data":"dc016e61200fe103e381595e413ef36be4e70f579a7c8ecc7c165022ad813465"}
Jul 06 23:57:39.840332 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.840307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5" event={"ID":"ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1","Type":"ContainerStarted","Data":"e69b456aaa84d9aed5202e1f86bffccac2d5ba53c8aff00ae4f1776a4cd7d737"}
Jul 06 23:57:39.841993 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.841972    2581 generic.go:356] "Generic (PLEG): container finished" podID="48b7d031-87ca-4fe0-b7ec-ae5554d05a7c" containerID="c1ee50aa4751a014af8fbfa3d6a1f77caa7e6bfcd17dfb1e6aa52d5168fe2a5c" exitCode=0
Jul 06 23:57:39.842087 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.842018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5gzj" event={"ID":"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c","Type":"ContainerDied","Data":"c1ee50aa4751a014af8fbfa3d6a1f77caa7e6bfcd17dfb1e6aa52d5168fe2a5c"}
Jul 06 23:57:39.862067 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.862013    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-4l7rc" podStartSLOduration=2.518706042 podStartE2EDuration="3.861992739s" totalImagesPullingTime="1.343286697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:39.861192382 +0000 UTC m=+62.264588481" watchObservedRunningTime="2026-07-06 23:57:39.861992739 +0000 UTC m=+62.265388814"
Jul 06 23:57:39.904734 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.904680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-stvj5" podStartSLOduration=2.480330453 podStartE2EDuration="3.904661207s" totalImagesPullingTime="1.424330754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:39.904164328 +0000 UTC m=+62.307560430" watchObservedRunningTime="2026-07-06 23:57:39.904661207 +0000 UTC m=+62.308057282"
Jul 06 23:57:39.962823 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:39.962788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:40.066669 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.066638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:40.128474 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.128391    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7788cc8d99-llz8p"]
Jul 06 23:57:40.149245 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.149211    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7788cc8d99-llz8p"]
Jul 06 23:57:40.149428 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.149377    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152818    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152835    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152921    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152931    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152835    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 06 23:57:40.153043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.152934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6lcmp\""
Jul 06 23:57:40.153748 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.153729    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bpft5nu3jbslr\""
Jul 06 23:57:40.198898 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.198838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.198914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e1111f-023f-4214-a572-941c0ed562e1-metrics-client-ca\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.198944    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.198978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8246s\" (UniqueName: \"kubernetes.io/projected/19e1111f-023f-4214-a572-941c0ed562e1-kube-api-access-8246s\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.199019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-grpc-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199105 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.199099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.199159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.199362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.199189    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.213341 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.213275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:40.300382 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300578 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300578 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e1111f-023f-4214-a572-941c0ed562e1-metrics-client-ca\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300578 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300578 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8246s\" (UniqueName: \"kubernetes.io/projected/19e1111f-023f-4214-a572-941c0ed562e1-kube-api-access-8246s\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300784 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-grpc-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300784 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.300784 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.300724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.302014 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.301989    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e1111f-023f-4214-a572-941c0ed562e1-metrics-client-ca\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.303786 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.303760    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.303904 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.303760    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.303904 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.303878    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-grpc-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.304410 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.304384    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.304561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.304541    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-tls\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.304561 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.304548    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/19e1111f-023f-4214-a572-941c0ed562e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.318891 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.318848    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8246s\" (UniqueName: \"kubernetes.io/projected/19e1111f-023f-4214-a572-941c0ed562e1-kube-api-access-8246s\") pod \"thanos-querier-7788cc8d99-llz8p\" (UID: \"19e1111f-023f-4214-a572-941c0ed562e1\") " pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.319507 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.319479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:40.460907 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.460814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:40.726481 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.726453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7788cc8d99-llz8p"]
Jul 06 23:57:40.729417 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:40.729392    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e1111f_023f_4214_a572_941c0ed562e1.slice/crio-f6a9543974c2c3a6db6eed535db6288f7a300c8dc827b5970c9980f637674d3f WatchSource:0}: Error finding container f6a9543974c2c3a6db6eed535db6288f7a300c8dc827b5970c9980f637674d3f: Status 404 returned error can't find the container with id f6a9543974c2c3a6db6eed535db6288f7a300c8dc827b5970c9980f637674d3f
Jul 06 23:57:40.847539 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.847504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5gzj" event={"ID":"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c","Type":"ContainerStarted","Data":"0905e0d0c29335c0185b6f338c67bf88e2494776c361122b52970600af926230"}
Jul 06 23:57:40.847976 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.847566    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5gzj" event={"ID":"48b7d031-87ca-4fe0-b7ec-ae5554d05a7c","Type":"ContainerStarted","Data":"fd103a8794fc7b7d75c4bfd19bebea578d022cf0f302bf5bd6781b3d673b395d"}
Jul 06 23:57:40.848768 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.848744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"f6a9543974c2c3a6db6eed535db6288f7a300c8dc827b5970c9980f637674d3f"}
Jul 06 23:57:40.850205 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.850186    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c" exitCode=0
Jul 06 23:57:40.850296 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.850275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c"}
Jul 06 23:57:40.872065 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:40.872020    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z5gzj" podStartSLOduration=2.218819666 podStartE2EDuration="3.872003296s" totalImagesPullingTime="1.540066816s" totalInitContainerRuntime="113.116814ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:40.871042432 +0000 UTC m=+63.274438520" watchObservedRunningTime="2026-07-06 23:57:40.872003296 +0000 UTC m=+63.275399392"
Jul 06 23:57:41.069116 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.069026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:41.376622 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.376529    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86c58d4676-7phlj"]
Jul 06 23:57:41.379327 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.379302    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.382284 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.382260    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 23:57:41.382761 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.382741    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 06 23:57:41.382892 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.382792    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 06 23:57:41.382892 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.382837    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-h4l8x\""
Jul 06 23:57:41.383889 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.383743    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 06 23:57:41.383889 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.383882    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-r360mdabekml\""
Jul 06 23:57:41.389397 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.389376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86c58d4676-7phlj"]
Jul 06 23:57:41.513285 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/012ead98-7d13-4ef0-8863-5ea867e01fca-audit-log\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-metrics-server-audit-profiles\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-client-certs\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-tls\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513663 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513520    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9vfw\" (UniqueName: \"kubernetes.io/projected/012ead98-7d13-4ef0-8863-5ea867e01fca-kube-api-access-q9vfw\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.513704 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.513671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-client-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615086 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/012ead98-7d13-4ef0-8863-5ea867e01fca-audit-log\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615086 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615327 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-metrics-server-audit-profiles\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615327 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-client-certs\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615327 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-tls\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615327 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9vfw\" (UniqueName: \"kubernetes.io/projected/012ead98-7d13-4ef0-8863-5ea867e01fca-kube-api-access-q9vfw\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615550 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-client-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.615550 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.615444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/012ead98-7d13-4ef0-8863-5ea867e01fca-audit-log\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.616255 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.616226    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.616440 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.616417    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/012ead98-7d13-4ef0-8863-5ea867e01fca-metrics-server-audit-profiles\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.618047 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.618024    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-client-ca-bundle\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.618397 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.618371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-tls\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.618562 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.618537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/012ead98-7d13-4ef0-8863-5ea867e01fca-secret-metrics-server-client-certs\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.625226 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.625186    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9vfw\" (UniqueName: \"kubernetes.io/projected/012ead98-7d13-4ef0-8863-5ea867e01fca-kube-api-access-q9vfw\") pod \"metrics-server-86c58d4676-7phlj\" (UID: \"012ead98-7d13-4ef0-8863-5ea867e01fca\") " pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.692162 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.692080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:41.694948 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.694916    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:57:41.696476 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.696389    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"]
Jul 06 23:57:41.699265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.699247    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.704997 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.702589    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-srbc2\""
Jul 06 23:57:41.704997 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.702828    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 06 23:57:41.709382 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.709322    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"]
Jul 06 23:57:41.817564 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.817530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7776f20b-0ce6-47fa-9229-06ceb3ba3130-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.817728 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.817605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgzlw\" (UniqueName: \"kubernetes.io/projected/7776f20b-0ce6-47fa-9229-06ceb3ba3130-kube-api-access-mgzlw\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.850592 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.850549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86c58d4676-7phlj"]
Jul 06 23:57:41.919299 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.919050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7776f20b-0ce6-47fa-9229-06ceb3ba3130-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.919484 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.919315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgzlw\" (UniqueName: \"kubernetes.io/projected/7776f20b-0ce6-47fa-9229-06ceb3ba3130-kube-api-access-mgzlw\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.922144 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.922121    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7776f20b-0ce6-47fa-9229-06ceb3ba3130-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.928897 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.928834    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgzlw\" (UniqueName: \"kubernetes.io/projected/7776f20b-0ce6-47fa-9229-06ceb3ba3130-kube-api-access-mgzlw\") pod \"monitoring-plugin-d5dcff7cf-wgzs6\" (UID: \"7776f20b-0ce6-47fa-9229-06ceb3ba3130\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:41.931761 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:41.931736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:42.027442 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.027408    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:42.177564 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.177379    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85db4fc479-lv25d"]
Jul 06 23:57:42.181109 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.180929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.184303 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.183952    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 06 23:57:42.184303 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.184007    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 06 23:57:42.184303 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.184108    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 06 23:57:42.184303 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.184012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 06 23:57:42.184303 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.183958    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 06 23:57:42.184622 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.184395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w9trm\""
Jul 06 23:57:42.189836 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.189812    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 06 23:57:42.193773 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.193751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85db4fc479-lv25d"]
Jul 06 23:57:42.228141 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:42.228107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012ead98_7d13_4ef0_8863_5ea867e01fca.slice/crio-ca637993465596406bd165886ca56aabf3ad13d6109b2048048308bf65ff3e2f WatchSource:0}: Error finding container ca637993465596406bd165886ca56aabf3ad13d6109b2048048308bf65ff3e2f: Status 404 returned error can't find the container with id ca637993465596406bd165886ca56aabf3ad13d6109b2048048308bf65ff3e2f
Jul 06 23:57:42.323611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wkxc\" (UniqueName: \"kubernetes.io/projected/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-kube-api-access-4wkxc\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-serving-certs-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-federate-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.323835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.323796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-metrics-client-ca\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425232 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wkxc\" (UniqueName: \"kubernetes.io/projected/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-kube-api-access-4wkxc\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-serving-certs-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-federate-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425691 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.425691 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.425483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-metrics-client-ca\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.426292 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.426258    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-metrics-client-ca\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.426422 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.426339    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-serving-certs-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.426775 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.426750    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.428323 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.428301    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.428509 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.428483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-federate-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.428611 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.428541    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-telemeter-client-tls\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.428818 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.428799    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.435068 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.435026    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wkxc\" (UniqueName: \"kubernetes.io/projected/bf496a13-facb-4e4d-8f0d-b4aa7b69eadd-kube-api-access-4wkxc\") pod \"telemeter-client-85db4fc479-lv25d\" (UID: \"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd\") " pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.497040 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.496998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d"
Jul 06 23:57:42.861518 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.861452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj" event={"ID":"012ead98-7d13-4ef0-8863-5ea867e01fca","Type":"ContainerStarted","Data":"ca637993465596406bd165886ca56aabf3ad13d6109b2048048308bf65ff3e2f"}
Jul 06 23:57:42.950329 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.949289    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"]
Jul 06 23:57:42.987992 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:42.987918    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85db4fc479-lv25d"]
Jul 06 23:57:42.994430 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:42.994295    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf496a13_facb_4e4d_8f0d_b4aa7b69eadd.slice/crio-0bd9ad6356922c958b93838564896558bd5f34b8abd847dfc7e3533da1eaa0e1 WatchSource:0}: Error finding container 0bd9ad6356922c958b93838564896558bd5f34b8abd847dfc7e3533da1eaa0e1: Status 404 returned error can't find the container with id 0bd9ad6356922c958b93838564896558bd5f34b8abd847dfc7e3533da1eaa0e1
Jul 06 23:57:43.140959 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.140884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:43.244340 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.244310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:43.352027 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.351949    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:43.357725 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.357691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.360045 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.360014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:43.362008 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.361988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:57:43.362140 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.362036    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:57:43.362198 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.362144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:57:43.362494 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.362475    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:57:43.362494 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.362480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wpp46\""
Jul 06 23:57:43.363559 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.363539    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:57:43.364108 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.364090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:57:43.364311 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.364295    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:57:43.365446 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.364764    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:57:43.366741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.365782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:57:43.366741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.365958    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fcg9c0qv92ocv\""
Jul 06 23:57:43.366741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.366016    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:57:43.366741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.366198    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:57:43.366741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.366371    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:57:43.367514 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.367235    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:57:43.376442 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.376420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:57:43.377196 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.377176    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:43.435573 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435735 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.435979 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.435936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgltw\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436208 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436441 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436441 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.436441 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.436306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.536836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgltw\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.536940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.537140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.537199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.537227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.537265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538032    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.538452 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.539006 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.539006 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.538527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.544604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.544360    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.545676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.545652    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547101 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.546254    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547101 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.546775    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547101 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.547057    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547940 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.547307    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547940 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.547380    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.547940 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.547464    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.548963 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.548763    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.549372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.549354    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.550268 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.550178    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.551043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.551024    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.551184 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.551137    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.551408 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.551384    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.551502 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.551482    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.551948 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.551906    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.552084 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.552050    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.553081 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.553039    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.553746 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.553725    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgltw\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw\") pod \"prometheus-k8s-0\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.673225 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.672920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:43.809645 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.809000    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7tsxx"
Jul 06 23:57:43.871272 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.871229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6" event={"ID":"7776f20b-0ce6-47fa-9229-06ceb3ba3130","Type":"ContainerStarted","Data":"da5549e2ce9833a0dcb57427d88447966136ae960848a85a0e6e2818f029e207"}
Jul 06 23:57:43.876664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.876592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d" event={"ID":"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd","Type":"ContainerStarted","Data":"0bd9ad6356922c958b93838564896558bd5f34b8abd847dfc7e3533da1eaa0e1"}
Jul 06 23:57:43.880252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.880224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"a6f90910ce68fa47ec954103ba882ded24af7186b9ae37277840753744f9c135"}
Jul 06 23:57:43.880388 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.880293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"25a406b50bc07d25ed19c1f577fe6dff46d6f3803d6b19ae76e48dabf3849a3b"}
Jul 06 23:57:43.880388 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.880332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"8a7b859ae33cbc718bd7d8aeb1a06531210901a824c2d8dfbf28e46fc24a7876"}
Jul 06 23:57:43.885238 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.885176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"}
Jul 06 23:57:43.885352 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.885245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"}
Jul 06 23:57:43.885352 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.885286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"}
Jul 06 23:57:43.885352 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.885335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"}
Jul 06 23:57:43.885521 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:43.885375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"}
Jul 06 23:57:44.249362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.249319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:44.249566 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.249398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:44.252254 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.252231    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:57:44.253257 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.253227    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:57:44.263372 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.263347    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63-original-pull-secret\") pod \"global-pull-secret-syncer-hp5lp\" (UID: \"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63\") " pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:44.263875 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.263840    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/704d581b-123b-42b3-a7d7-176251a58865-metrics-certs\") pod \"network-metrics-daemon-s7xxc\" (UID: \"704d581b-123b-42b3-a7d7-176251a58865\") " pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:44.350526 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.350486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:44.353835 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.353805    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.364598 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.364577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.370290 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.370263    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-knf8z\""
Jul 06 23:57:44.374810 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.374783    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-249np\" (UniqueName: \"kubernetes.io/projected/4ca593e8-3b94-4801-b466-70d71346e978-kube-api-access-249np\") pod \"network-check-target-6llz8\" (UID: \"4ca593e8-3b94-4801-b466-70d71346e978\") " pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:44.377964 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.377939    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s7xxc"
Jul 06 23:57:44.378071 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.378059    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lswch\""
Jul 06 23:57:44.383446 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.383421    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp5lp"
Jul 06 23:57:44.385574 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:44.385521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:45.382048 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.381773    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hp5lp"]
Jul 06 23:57:45.394464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.392335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s7xxc"]
Jul 06 23:57:45.395390 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:45.395321    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8ea7e7_5219_4e2b_a7ee_cdf90cd72c63.slice/crio-b4d71e073b58c81625f48adf4f7209fc447a671010709c3999d6c78c86abcb65 WatchSource:0}: Error finding container b4d71e073b58c81625f48adf4f7209fc447a671010709c3999d6c78c86abcb65: Status 404 returned error can't find the container with id b4d71e073b58c81625f48adf4f7209fc447a671010709c3999d6c78c86abcb65
Jul 06 23:57:45.408738 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.406027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:57:45.450418 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.449991    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6llz8"]
Jul 06 23:57:45.454877 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:57:45.454836    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca593e8_3b94_4801_b466_70d71346e978.slice/crio-bdcc458b75bee0e3c57a77faf4d3455740f90a1bce0e12f6b69f9f92226880ab WatchSource:0}: Error finding container bdcc458b75bee0e3c57a77faf4d3455740f90a1bce0e12f6b69f9f92226880ab: Status 404 returned error can't find the container with id bdcc458b75bee0e3c57a77faf4d3455740f90a1bce0e12f6b69f9f92226880ab
Jul 06 23:57:45.895976 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.895945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7xxc" event={"ID":"704d581b-123b-42b3-a7d7-176251a58865","Type":"ContainerStarted","Data":"abded0653e3f4fa2eefd5d7ba40ff6716f8132ede86ce4d27703e4a67cce3aa0"}
Jul 06 23:57:45.905885 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.902345    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"04f2a3319a70b653470fd792b1b55f7833248250425fc08ce54644f60858dc4f"}
Jul 06 23:57:45.905885 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.902494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"c4f75b70b697118e38ddf79390bdb6592df5dbcdbf1f97b9cfddda846e2635b0"}
Jul 06 23:57:45.905885 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.902554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" event={"ID":"19e1111f-023f-4214-a572-941c0ed562e1","Type":"ContainerStarted","Data":"1e05379ddccdc5a319f026ee635d39399b7616834ae8f21d54458e807b765ac0"}
Jul 06 23:57:45.905885 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.903059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:45.908558 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.908338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6llz8" event={"ID":"4ca593e8-3b94-4801-b466-70d71346e978","Type":"ContainerStarted","Data":"bdcc458b75bee0e3c57a77faf4d3455740f90a1bce0e12f6b69f9f92226880ab"}
Jul 06 23:57:45.910124 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.910096    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hp5lp" event={"ID":"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63","Type":"ContainerStarted","Data":"b4d71e073b58c81625f48adf4f7209fc447a671010709c3999d6c78c86abcb65"}
Jul 06 23:57:45.915311 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.915282    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerStarted","Data":"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"}
Jul 06 23:57:45.918159 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.918134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj" event={"ID":"012ead98-7d13-4ef0-8863-5ea867e01fca","Type":"ContainerStarted","Data":"632747b95f880aa8356861be71bb58067ba481150cbe37bf49d00d7b990a21b9"}
Jul 06 23:57:45.920433 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.920405    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6" event={"ID":"7776f20b-0ce6-47fa-9229-06ceb3ba3130","Type":"ContainerStarted","Data":"8f02a7097d91adb641cae9e0304b13e775185d0eb7a4b4b7fce68bf6d1f5d51c"}
Jul 06 23:57:45.921121 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.921104    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:45.925013 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.924995    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05" exitCode=0
Jul 06 23:57:45.925209 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.925076    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05"}
Jul 06 23:57:45.925390 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.925375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"fe5d22f523847e1666d3ee5e901b7b4ebdeba1b29c58f758c5b1a5c2d9f7da3c"}
Jul 06 23:57:45.927381 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.927324    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p" podStartSLOduration=1.803094752 podStartE2EDuration="5.92730835s" totalImagesPullingTime="4.124213598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:45.926891456 +0000 UTC m=+68.330287531" watchObservedRunningTime="2026-07-06 23:57:45.92730835 +0000 UTC m=+68.330704428"
Jul 06 23:57:45.928166 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.928148    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6"
Jul 06 23:57:45.931518 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.931491    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d" event={"ID":"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd","Type":"ContainerStarted","Data":"e157d110c8ae1679da6db53d6f3c19f5d3a79b8f84404043773a9e19d9f7212c"}
Jul 06 23:57:45.931604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.931542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d" event={"ID":"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd","Type":"ContainerStarted","Data":"f0a0cf8fb4715c642a33e35a30c57a455a9fec121d6249541c7bae1a8079f7fa"}
Jul 06 23:57:45.931604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.931564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d" event={"ID":"bf496a13-facb-4e4d-8f0d-b4aa7b69eadd","Type":"ContainerStarted","Data":"d933b55b8f020e731ceb6def73c9078c8363b050c107697e9a1f92ad3c3d9d24"}
Jul 06 23:57:45.945004 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.944942    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj" podStartSLOduration=2.020015635 podStartE2EDuration="4.944923543s" totalImagesPullingTime="2.924907908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:45.944772963 +0000 UTC m=+68.348169038" watchObservedRunningTime="2026-07-06 23:57:45.944923543 +0000 UTC m=+68.348319617"
Jul 06 23:57:45.966574 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:45.966517    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-wgzs6" podStartSLOduration=2.742538037 podStartE2EDuration="4.966499041s" totalImagesPullingTime="2.223961004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:45.965976254 +0000 UTC m=+68.369372328" watchObservedRunningTime="2026-07-06 23:57:45.966499041 +0000 UTC m=+68.369895114"
Jul 06 23:57:46.075938 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:46.075878    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.866407857 podStartE2EDuration="8.075838994s" totalImagesPullingTime="5.185990483s" totalInitContainerRuntime="23.440654ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:46.075209145 +0000 UTC m=+68.478605220" watchObservedRunningTime="2026-07-06 23:57:46.075838994 +0000 UTC m=+68.479235068"
Jul 06 23:57:46.121954 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:46.121892    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85db4fc479-lv25d" podStartSLOduration=1.859105511 podStartE2EDuration="4.121874248s" totalImagesPullingTime="2.262768737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:46.120058794 +0000 UTC m=+68.523454868" watchObservedRunningTime="2026-07-06 23:57:46.121874248 +0000 UTC m=+68.525270318"
Jul 06 23:57:47.674473 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.674387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:47.771283 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.771249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:47.871429 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.871360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:47.944822 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.944787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7xxc" event={"ID":"704d581b-123b-42b3-a7d7-176251a58865","Type":"ContainerStarted","Data":"8c7fadb10e941051f5d102af863aab42e1ec38b1a6e7e43dfc580ef6fde024c0"}
Jul 06 23:57:47.945082 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.944905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s7xxc" event={"ID":"704d581b-123b-42b3-a7d7-176251a58865","Type":"ContainerStarted","Data":"33441b5f2c90209a5175b8cd553f444ad40a26101c79f24a17c188c7c6a8c36b"}
Jul 06 23:57:47.971103 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:47.971037    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s7xxc" podStartSLOduration=68.229152429 podStartE2EDuration="1m9.971014523s" totalImagesPullingTime="1.741862094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:47.968477777 +0000 UTC m=+70.371873892" watchObservedRunningTime="2026-07-06 23:57:47.971014523 +0000 UTC m=+70.374410598"
Jul 06 23:57:48.105926 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.105658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.175239 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.175203    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.279212 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.279179    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.375908 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.375698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.576691 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.576645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.762607 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.762543    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.824162 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:48.824122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.147256 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:49.147176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.247748 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:49.247069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.348507 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:49.348471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.448210 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:49.448125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.749660 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:49.749609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.217533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:50.217476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.986738 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:50.986710    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.086891 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.086848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.947659 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.947325    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7788cc8d99-llz8p"
Jul 06 23:57:51.973722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.973015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6llz8" event={"ID":"4ca593e8-3b94-4801-b466-70d71346e978","Type":"ContainerStarted","Data":"82318db188667808ae2ca3991e12156210cd20f6ebeebc628d1f3b1de5736bc1"}
Jul 06 23:57:51.973722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.973607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:57:51.975943 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.975850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hp5lp" event={"ID":"5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63","Type":"ContainerStarted","Data":"c0c4b2121ce3bdfe25b2f114dd826fca28281742548809a4ab997f2ae8929678"}
Jul 06 23:57:51.981440 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.981384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"}
Jul 06 23:57:51.981574 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.981467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"}
Jul 06 23:57:51.981574 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:51.981509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"}
Jul 06 23:57:52.006930 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.006236    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6llz8" podStartSLOduration=67.7702063 podStartE2EDuration="1m14.006215058s" totalImagesPullingTime="6.236008758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:52.003978352 +0000 UTC m=+74.407374473" watchObservedRunningTime="2026-07-06 23:57:52.006215058 +0000 UTC m=+74.409611133"
Jul 06 23:57:52.021901 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.021813    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hp5lp" podStartSLOduration=67.709843013 podStartE2EDuration="1m14.021791818s" totalImagesPullingTime="6.311948805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:52.019842269 +0000 UTC m=+74.423238368" watchObservedRunningTime="2026-07-06 23:57:52.021791818 +0000 UTC m=+74.425187891"
Jul 06 23:57:52.127806 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.127781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:52.229084 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.229050    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:52.941177 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.941142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:52.988382 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.988348    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"}
Jul 06 23:57:52.988759 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.988411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"}
Jul 06 23:57:52.988759 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:52.988439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerStarted","Data":"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"}
Jul 06 23:57:53.007115 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:53.007087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:53.021487 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:53.021442    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.225421697 podStartE2EDuration="10.021426779s" totalImagesPullingTime="5.770356016s" totalInitContainerRuntime="25.649066ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:53.018929141 +0000 UTC m=+75.422325235" watchObservedRunningTime="2026-07-06 23:57:53.021426779 +0000 UTC m=+75.424822883"
Jul 06 23:57:53.673563 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:53.673529    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:57:56.103516 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:56.103483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:56.158595 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:57:56.158561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.695916 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:01.695744    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:58:01.695916 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:01.695878    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:58:02.969911 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:02.969878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:03.040091 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:03.040059    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.769937 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:06.769901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.833932 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:06.833903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:10.169732 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:10.169690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:11.101071 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:11.101043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:11.204173 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:11.204139    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.398173 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:16.398138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.486364 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:16.486338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.579827 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:16.579794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.644931 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:16.644900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:17.002189 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:17.002149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:21.700748 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:21.700706    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:58:21.704583 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:21.704562    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86c58d4676-7phlj"
Jul 06 23:58:23.995638 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:23.995614    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6llz8"
Jul 06 23:58:26.101660 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:26.101619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:26.169804 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:26.169770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:31.914299 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:31.914213    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.590873 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:43.590833    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.673701 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:43.673664    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:43.692539 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:43.692515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:44.221060 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:44.221032    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:58:53.158950 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:53.158914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:53.235968 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:53.235937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:57.403367 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403330    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:57.403781 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403632    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="alertmanager" containerID="cri-o://e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6" gracePeriod=120
Jul 06 23:58:57.403781 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403736    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-web" containerID="cri-o://7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007" gracePeriod=120
Jul 06 23:58:57.403781 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403751    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy" containerID="cri-o://2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186" gracePeriod=120
Jul 06 23:58:57.404024 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403707    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="prom-label-proxy" containerID="cri-o://b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc" gracePeriod=120
Jul 06 23:58:57.404024 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403804    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="config-reloader" containerID="cri-o://979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5" gracePeriod=120
Jul 06 23:58:57.404024 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:57.403906    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-metric" containerID="cri-o://a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720" gracePeriod=120
Jul 06 23:58:58.267421 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267387    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc" exitCode=0
Jul 06 23:58:58.267421 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267415    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720" exitCode=0
Jul 06 23:58:58.267421 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267421    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186" exitCode=0
Jul 06 23:58:58.267421 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267427    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5" exitCode=0
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267432    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6" exitCode=0
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"}
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"}
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"}
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"}
Jul 06 23:58:58.267676 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.267595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"}
Jul 06 23:58:58.641375 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.641350    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:58.720842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720874    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720949    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.720981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721032    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721113    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9l52\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets\") pod \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\" (UID: \"7c3f6b14-64cb-48b2-a97a-c58bb7856906\") "
Jul 06 23:58:58.721324 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721284    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:58.721654 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721499    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-main-db\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.722019 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.721725    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:58.722335 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.722297    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:58.724338 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.724292    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.724712 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.724685    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out" (OuterVolumeSpecName: "config-out") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:58.724806 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.724745    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52" (OuterVolumeSpecName: "kube-api-access-x9l52") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "kube-api-access-x9l52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:58.725063 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.725029    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.725186 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.725147    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.725333 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.725304    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.725394 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.725359    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:58.725761 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.725740    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume" (OuterVolumeSpecName: "config-volume") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.729356 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.729337    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.736119 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.736093    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config" (OuterVolumeSpecName: "web-config") pod "7c3f6b14-64cb-48b2-a97a-c58bb7856906" (UID: "7c3f6b14-64cb-48b2-a97a-c58bb7856906"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821851    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-cluster-tls-config\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821896    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821908    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821917    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-main-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821926    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-volume\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821934    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c3f6b14-64cb-48b2-a97a-c58bb7856906-config-out\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821941    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.821944 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821950    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.822278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821959    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c3f6b14-64cb-48b2-a97a-c58bb7856906-web-config\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.822278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821967    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c3f6b14-64cb-48b2-a97a-c58bb7856906-metrics-client-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.822278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821976    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9l52\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-kube-api-access-x9l52\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:58.822278 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:58.821983    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c3f6b14-64cb-48b2-a97a-c58bb7856906-tls-assets\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:58:59.278883 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.278825    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerID="7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007" exitCode=0
Jul 06 23:58:59.278883 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.278882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"}
Jul 06 23:58:59.279133 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.278939    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.279133 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.278981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7c3f6b14-64cb-48b2-a97a-c58bb7856906","Type":"ContainerDied","Data":"403dba610d2024384f00712dedc6cbba9648bfcd01724358bca1a521c44a1e4e"}
Jul 06 23:58:59.279133 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.279041    2581 scope.go:117] "RemoveContainer" containerID="b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"
Jul 06 23:58:59.287312 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.287295    2581 scope.go:117] "RemoveContainer" containerID="a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"
Jul 06 23:58:59.294149 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.294134    2581 scope.go:117] "RemoveContainer" containerID="2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"
Jul 06 23:58:59.301156 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.301129    2581 scope.go:117] "RemoveContainer" containerID="7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"
Jul 06 23:58:59.303485 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.303461    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:59.308009 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.307985    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:59.310028 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.310011    2581 scope.go:117] "RemoveContainer" containerID="979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"
Jul 06 23:58:59.317054 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.317032    2581 scope.go:117] "RemoveContainer" containerID="e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"
Jul 06 23:58:59.324081 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.324063    2581 scope.go:117] "RemoveContainer" containerID="e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c"
Jul 06 23:58:59.331441 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.331422    2581 scope.go:117] "RemoveContainer" containerID="b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"
Jul 06 23:58:59.331842 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.331815    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc\": container with ID starting with b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc not found: ID does not exist" containerID="b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"
Jul 06 23:58:59.332016 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.331873    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc"} err="failed to get container status \"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc\": rpc error: code = NotFound desc = could not find container \"b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc\": container with ID starting with b6d5001aaf9ad97f18821d54e84a6658c89be61f9f5d636395f3f52d1ef35efc not found: ID does not exist"
Jul 06 23:58:59.332016 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.331919    2581 scope.go:117] "RemoveContainer" containerID="a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"
Jul 06 23:58:59.332246 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.332226    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720\": container with ID starting with a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720 not found: ID does not exist" containerID="a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"
Jul 06 23:58:59.332295 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332254    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720"} err="failed to get container status \"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720\": rpc error: code = NotFound desc = could not find container \"a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720\": container with ID starting with a7f3db4bd16027ed61fcb117e6225369bcb8bce95a9835753d3afa16b784d720 not found: ID does not exist"
Jul 06 23:58:59.332295 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332271    2581 scope.go:117] "RemoveContainer" containerID="2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"
Jul 06 23:58:59.332487 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.332471    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186\": container with ID starting with 2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186 not found: ID does not exist" containerID="2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"
Jul 06 23:58:59.332541 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332489    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186"} err="failed to get container status \"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186\": rpc error: code = NotFound desc = could not find container \"2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186\": container with ID starting with 2825f00e4312a5ee277239dc510d6d5c032d88a96094e115365b379152d8c186 not found: ID does not exist"
Jul 06 23:58:59.332541 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332501    2581 scope.go:117] "RemoveContainer" containerID="7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"
Jul 06 23:58:59.332713 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332698    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:59.332713 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.332706    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007\": container with ID starting with 7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007 not found: ID does not exist" containerID="7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"
Jul 06 23:58:59.332779 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332724    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007"} err="failed to get container status \"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007\": rpc error: code = NotFound desc = could not find container \"7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007\": container with ID starting with 7969be85bacb5cee36a8dcd7ec1bc8b50e46c116cce64da42e79deb2cabf5007 not found: ID does not exist"
Jul 06 23:58:59.332779 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.332740    2581 scope.go:117] "RemoveContainer" containerID="979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"
Jul 06 23:58:59.333069 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.333043    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5\": container with ID starting with 979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5 not found: ID does not exist" containerID="979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"
Jul 06 23:58:59.333069 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333062    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5"} err="failed to get container status \"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5\": rpc error: code = NotFound desc = could not find container \"979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5\": container with ID starting with 979adfd11462bff0b5c947ea2eb9466e2c7d86503a61198cd139e11fee0ff7b5 not found: ID does not exist"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333076    2581 scope.go:117] "RemoveContainer" containerID="e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333076    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-web"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333121    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-web"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333132    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333137    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333155    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="init-config-reloader"
Jul 06 23:58:59.333171 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333161    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="init-config-reloader"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333178    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="config-reloader"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333183    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="config-reloader"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333191    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="alertmanager"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333196    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="alertmanager"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333224    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333234    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333250    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="prom-label-proxy"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333260    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="prom-label-proxy"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.333315    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6\": container with ID starting with e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6 not found: ID does not exist" containerID="e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333333    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-web"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333343    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333357    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="alertmanager"
Jul 06 23:58:59.333362 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333363    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="prom-label-proxy"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333370    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="config-reloader"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333377    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" containerName="kube-rbac-proxy-metric"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333336    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6"} err="failed to get container status \"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6\": rpc error: code = NotFound desc = could not find container \"e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6\": container with ID starting with e7b61e706fe39774c8ecde70444065d55f253194aebe1de32e4abe3ca67038c6 not found: ID does not exist"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333425    2581 scope.go:117] "RemoveContainer" containerID="e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:58:59.333669    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c\": container with ID starting with e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c not found: ID does not exist" containerID="e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c"
Jul 06 23:58:59.333722 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.333694    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c"} err="failed to get container status \"e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c\": rpc error: code = NotFound desc = could not find container \"e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c\": container with ID starting with e878a718ea7fb28a16e50fa688aa4bb3394ad1166b915d632c7d9d5b9459e47c not found: ID does not exist"
Jul 06 23:58:59.338138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.338118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.342485 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.342465    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 23:58:59.342644 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.342623    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znb9k\""
Jul 06 23:58:59.342720 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.342704    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 23:58:59.342972 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.342951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 23:58:59.343061 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.342959    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 23:58:59.343126 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.343087    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 23:58:59.343333 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.343297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 23:58:59.343333 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.343327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 23:58:59.343459 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.343351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 23:58:59.348476 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.348457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 23:58:59.354761 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.354742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:59.427138 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427308 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-web-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-249gw\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-kube-api-access-249gw\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-config-volume\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-config-out\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.427501 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.427439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528441 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528566 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-web-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528566 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528643 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249gw\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-kube-api-access-249gw\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-config-volume\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-config-out\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.528754 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.528683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529107 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529107 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529101    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529265 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529414 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.529635 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.529608    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.530462 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.530432    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.530876 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.530839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65644a99-09ac-4b14-9fa3-c1d786556999-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.531634 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.531609    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-tls-assets\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.531719 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.531647    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.532061 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.532018    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65644a99-09ac-4b14-9fa3-c1d786556999-config-out\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.532163 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.532060    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.532230 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.532192    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-config-volume\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.532637 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.532613    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.532715 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.532661    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-web-config\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.533239 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.533215    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.533523 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.533507    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/65644a99-09ac-4b14-9fa3-c1d786556999-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.538533 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.538514    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-249gw\" (UniqueName: \"kubernetes.io/projected/65644a99-09ac-4b14-9fa3-c1d786556999-kube-api-access-249gw\") pod \"alertmanager-main-0\" (UID: \"65644a99-09ac-4b14-9fa3-c1d786556999\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.647061 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.647011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 23:58:59.797211 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:58:59.797188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 23:58:59.799600 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:58:59.799572    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65644a99_09ac_4b14_9fa3_c1d786556999.slice/crio-3d78f010d0cdf5989c523aae088edffc46e704dfa85d5a731bbdf430ae17cb2e WatchSource:0}: Error finding container 3d78f010d0cdf5989c523aae088edffc46e704dfa85d5a731bbdf430ae17cb2e: Status 404 returned error can't find the container with id 3d78f010d0cdf5989c523aae088edffc46e704dfa85d5a731bbdf430ae17cb2e
Jul 06 23:59:00.283831 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.283794    2581 generic.go:356] "Generic (PLEG): container finished" podID="65644a99-09ac-4b14-9fa3-c1d786556999" containerID="e4652626a3a5a520a3c4c77785e2f62f907c3e44706c80c8984ed234d51f908e" exitCode=0
Jul 06 23:59:00.284043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.283882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerDied","Data":"e4652626a3a5a520a3c4c77785e2f62f907c3e44706c80c8984ed234d51f908e"}
Jul 06 23:59:00.284043 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.283961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"3d78f010d0cdf5989c523aae088edffc46e704dfa85d5a731bbdf430ae17cb2e"}
Jul 06 23:59:00.557495 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.557472    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c3f6b14-64cb-48b2-a97a-c58bb7856906" path="/var/lib/kubelet/pods/7c3f6b14-64cb-48b2-a97a-c58bb7856906/volumes"
Jul 06 23:59:00.766604 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.766568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:00.833895 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.833805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:00.895529 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:00.895495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:01.297498 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"860e7136c7a388bf3aac1dceddafa04f294f0e6fe590f025c5245a00ab254cee"}
Jul 06 23:59:01.297675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"5d8fcb917dfe4059d31d4d1f88e93addc803797402b62adc84e52d76c4c91781"}
Jul 06 23:59:01.297675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"59a33b7de4451a6bda84d0d501fc244d6e7b4813f5b2ead66aaab5628821ac30"}
Jul 06 23:59:01.297675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"df09bb54b308c1f2fd96a9c62ee65cc193067c24190436960145b62f4bb20c2b"}
Jul 06 23:59:01.297675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"73980c5c5338dd4bc9e3aca3a19d88cc987ac0a9fccc843f7ff6d58fe1000a6b"}
Jul 06 23:59:01.297675 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.297663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"65644a99-09ac-4b14-9fa3-c1d786556999","Type":"ContainerStarted","Data":"cf74eefa8990f4e6bc31d2ad2a64fba4343fbc0a2b00afbabd793b3b15475e7e"}
Jul 06 23:59:01.328921 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.328847    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.315567525 podStartE2EDuration="2.328825256s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.257731ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:01.328532379 +0000 UTC m=+143.731928486" watchObservedRunningTime="2026-07-06 23:59:01.328825256 +0000 UTC m=+143.732221330"
Jul 06 23:59:01.495448 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.495404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:01.595264 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.595187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:01.668283 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668243    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:01.668657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668618    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496" gracePeriod=600
Jul 06 23:59:01.668741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668672    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="thanos-sidecar" containerID="cri-o://5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd" gracePeriod=600
Jul 06 23:59:01.668741 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668695    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="config-reloader" containerID="cri-o://57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70" gracePeriod=600
Jul 06 23:59:01.668842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668651    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy" containerID="cri-o://6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25" gracePeriod=600
Jul 06 23:59:01.668842 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668612    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="prometheus" containerID="cri-o://bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e" gracePeriod=600
Jul 06 23:59:01.668967 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:01.668639    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-web" containerID="cri-o://788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7" gracePeriod=600
Jul 06 23:59:02.305451 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305408    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496" exitCode=0
Jul 06 23:59:02.305451 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305433    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25" exitCode=0
Jul 06 23:59:02.305451 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305439    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd" exitCode=0
Jul 06 23:59:02.305451 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305453    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70" exitCode=0
Jul 06 23:59:02.305451 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305459    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e" exitCode=0
Jul 06 23:59:02.305975 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"}
Jul 06 23:59:02.305975 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"}
Jul 06 23:59:02.305975 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305567    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"}
Jul 06 23:59:02.305975 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"}
Jul 06 23:59:02.305975 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.305621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"}
Jul 06 23:59:02.914986 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.914963    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:02.956634 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956644    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956702    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956738    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.956790 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956781    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956804    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956879    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956923    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgltw\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.956971    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957148    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957182    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957209    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0\") pod \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\" (UID: \"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5\") "
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957244    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957539    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-metrics-client-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:02.957780 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.957559    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:02.959413 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.959384    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:59:02.959413 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.959402    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.959742 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.959716    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.960477 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.960450    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out" (OuterVolumeSpecName: "config-out") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:59:02.962810 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.960844    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.962810 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.961144    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:59:02.962810 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.961272    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.963530 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.963400    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.963530 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.963444    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:02.963815 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.963779    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.964026 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.963970    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.964485 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.964460    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.965606 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.964557    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.965606 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.965077    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config" (OuterVolumeSpecName: "config") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.965815 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.965773    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.966214 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.966190    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw" (OuterVolumeSpecName: "kube-api-access-jgltw") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "kube-api-access-jgltw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:59:02.966280 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.966210    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:02.978750 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:02.978723    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config" (OuterVolumeSpecName: "web-config") pod "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" (UID: "b78d332e-fe4b-4d9b-bc1f-560995f1cfd5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058234    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-grpc-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058267    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgltw\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-kube-api-access-jgltw\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058278    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058292    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-kube-rbac-proxy\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058301    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-web-config\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058309    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-db\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058317    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058321 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058326    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058335    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058345    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058354    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058363    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config-out\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058370    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-tls-assets\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058378    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-metrics-client-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058386    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058394    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-config\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058402    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.058657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.058411    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.312541 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.312462    2581 generic.go:356] "Generic (PLEG): container finished" podID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerID="788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7" exitCode=0
Jul 06 23:59:03.312541 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.312507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"}
Jul 06 23:59:03.312541 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.312540    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.313031 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.312581    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b78d332e-fe4b-4d9b-bc1f-560995f1cfd5","Type":"ContainerDied","Data":"fe5d22f523847e1666d3ee5e901b7b4ebdeba1b29c58f758c5b1a5c2d9f7da3c"}
Jul 06 23:59:03.313031 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.312616    2581 scope.go:117] "RemoveContainer" containerID="8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"
Jul 06 23:59:03.320399 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.320276    2581 scope.go:117] "RemoveContainer" containerID="6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"
Jul 06 23:59:03.328148 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.328129    2581 scope.go:117] "RemoveContainer" containerID="788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"
Jul 06 23:59:03.334813 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.334794    2581 scope.go:117] "RemoveContainer" containerID="5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"
Jul 06 23:59:03.340164 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.340139    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:03.343096 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.343044    2581 scope.go:117] "RemoveContainer" containerID="57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"
Jul 06 23:59:03.344263 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.344244    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:03.349756 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.349738    2581 scope.go:117] "RemoveContainer" containerID="bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"
Jul 06 23:59:03.356989 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.356973    2581 scope.go:117] "RemoveContainer" containerID="7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05"
Jul 06 23:59:03.363692 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.363672    2581 scope.go:117] "RemoveContainer" containerID="8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"
Jul 06 23:59:03.363939 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.363919    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496\": container with ID starting with 8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496 not found: ID does not exist" containerID="8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"
Jul 06 23:59:03.363999 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.363948    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496"} err="failed to get container status \"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496\": rpc error: code = NotFound desc = could not find container \"8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496\": container with ID starting with 8ed6165a9fd50a310669ec0811b17676882d2a75d62daec9789014fbe0228496 not found: ID does not exist"
Jul 06 23:59:03.363999 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.363966    2581 scope.go:117] "RemoveContainer" containerID="6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"
Jul 06 23:59:03.364212 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.364193    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25\": container with ID starting with 6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25 not found: ID does not exist" containerID="6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"
Jul 06 23:59:03.364260 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364215    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25"} err="failed to get container status \"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25\": rpc error: code = NotFound desc = could not find container \"6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25\": container with ID starting with 6ff6b7eea98b93c30fc7a45da95699c74117b84d63a602676582001754ee8a25 not found: ID does not exist"
Jul 06 23:59:03.364260 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364230    2581 scope.go:117] "RemoveContainer" containerID="788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"
Jul 06 23:59:03.364433 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.364416    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7\": container with ID starting with 788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7 not found: ID does not exist" containerID="788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"
Jul 06 23:59:03.364471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364436    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7"} err="failed to get container status \"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7\": rpc error: code = NotFound desc = could not find container \"788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7\": container with ID starting with 788cf083b8ac4649ffd547fe0d46c96d95e578dff5538448e4de5a4df737c5d7 not found: ID does not exist"
Jul 06 23:59:03.364471 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364448    2581 scope.go:117] "RemoveContainer" containerID="5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"
Jul 06 23:59:03.364662 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.364645    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd\": container with ID starting with 5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd not found: ID does not exist" containerID="5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"
Jul 06 23:59:03.364709 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364664    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd"} err="failed to get container status \"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd\": rpc error: code = NotFound desc = could not find container \"5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd\": container with ID starting with 5cbdcca36258ab50db8b4488f5499821776c0910fb9010533b5b24f7ee5c35cd not found: ID does not exist"
Jul 06 23:59:03.364709 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364677    2581 scope.go:117] "RemoveContainer" containerID="57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"
Jul 06 23:59:03.364883 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.364843    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70\": container with ID starting with 57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70 not found: ID does not exist" containerID="57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"
Jul 06 23:59:03.364935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364882    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70"} err="failed to get container status \"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70\": rpc error: code = NotFound desc = could not find container \"57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70\": container with ID starting with 57c89f260c67c3299ff7853393ad0e585825a2e2d9465603d7a6720bd2be4e70 not found: ID does not exist"
Jul 06 23:59:03.364935 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.364901    2581 scope.go:117] "RemoveContainer" containerID="bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"
Jul 06 23:59:03.365140 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.365123    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e\": container with ID starting with bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e not found: ID does not exist" containerID="bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"
Jul 06 23:59:03.365184 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.365151    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e"} err="failed to get container status \"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e\": rpc error: code = NotFound desc = could not find container \"bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e\": container with ID starting with bddd2f4c42c0384fef6d1a3a4de3de9058d249536fa598a9a1f4747a23c9e17e not found: ID does not exist"
Jul 06 23:59:03.365184 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.365167    2581 scope.go:117] "RemoveContainer" containerID="7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05"
Jul 06 23:59:03.365415 ip-10-0-129-135 kubenswrapper[2581]: E0706 23:59:03.365397    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05\": container with ID starting with 7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05 not found: ID does not exist" containerID="7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05"
Jul 06 23:59:03.365474 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.365422    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05"} err="failed to get container status \"7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05\": rpc error: code = NotFound desc = could not find container \"7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05\": container with ID starting with 7624487f0fc0584b6d81d31d62136d6f366d8af3bb10825505c8c8b65c917a05 not found: ID does not exist"
Jul 06 23:59:03.370244 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370221    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:03.370582 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370566    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="init-config-reloader"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370585    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="init-config-reloader"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370597    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="prometheus"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370605    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="prometheus"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370625    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-thanos"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370634    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-thanos"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370655    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="thanos-sidecar"
Jul 06 23:59:03.370664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370663    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="thanos-sidecar"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370675    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370683    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370699    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="config-reloader"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370707    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="config-reloader"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370716    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-web"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370725    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-web"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370798    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="thanos-sidecar"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370809    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370823    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="prometheus"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370833    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-web"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370844    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="kube-rbac-proxy-thanos"
Jul 06 23:59:03.371005 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.370907    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" containerName="config-reloader"
Jul 06 23:59:03.374886 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.374851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.378570 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378548    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 23:59:03.378689 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378665    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 23:59:03.378771 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 23:59:03.378771 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 23:59:03.378771 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378620    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 23:59:03.378953 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378628    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wpp46\""
Jul 06 23:59:03.378953 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378560    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 23:59:03.379047 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.378990    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 23:59:03.379047 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.379018    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fcg9c0qv92ocv\""
Jul 06 23:59:03.379447 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.379430    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 23:59:03.379543 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.379508    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 23:59:03.379610 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.379542    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 23:59:03.379653 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.379607    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 23:59:03.382945 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.382789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 23:59:03.386550 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.386527    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 23:59:03.388304 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.388283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 23:59:03.389851 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.389827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:03.462444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462444 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462464    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v4p4\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-kube-api-access-8v4p4\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462666 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.462984 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.463252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.462997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.463252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.463020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.463252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.463042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.463252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.463071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.463252 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.463095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.563795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.563795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.563795 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4p4\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-kube-api-access-8v4p4\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.563976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564092 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564464 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564713 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564543    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564783 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564743    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.564877 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.564837    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.565765 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.565738    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.566590 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.566555    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.566989    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.566996    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567108    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567187    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567055    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567391 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567392    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567887 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.567887 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.567716    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.568123 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.568088    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.568466 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.568440    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.568789 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.568767    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.569312 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.569293    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.569412 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.569375    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.569502 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.569483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5bf10432-8644-4a6c-a0ef-3708bee2e33a-config\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.570145 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.570121    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5bf10432-8644-4a6c-a0ef-3708bee2e33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.572232 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.572215    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4p4\" (UniqueName: \"kubernetes.io/projected/5bf10432-8644-4a6c-a0ef-3708bee2e33a-kube-api-access-8v4p4\") pod \"prometheus-k8s-0\" (UID: \"5bf10432-8644-4a6c-a0ef-3708bee2e33a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.691103 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.691061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:03.867972 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:03.867880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 23:59:03.871610 ip-10-0-129-135 kubenswrapper[2581]: W0706 23:59:03.871583    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf10432_8644_4a6c_a0ef_3708bee2e33a.slice/crio-91e0c8854f570d9cae2dc7ed681dec9ce1e8a05a1726780f9b13fea55795d51d WatchSource:0}: Error finding container 91e0c8854f570d9cae2dc7ed681dec9ce1e8a05a1726780f9b13fea55795d51d: Status 404 returned error can't find the container with id 91e0c8854f570d9cae2dc7ed681dec9ce1e8a05a1726780f9b13fea55795d51d
Jul 06 23:59:04.318746 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:04.318709    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bf10432-8644-4a6c-a0ef-3708bee2e33a" containerID="2e3bc7abf512b8815dbb25c41b92caed41386b21569b17087b0d4d33cae70a04" exitCode=0
Jul 06 23:59:04.319232 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:04.318804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerDied","Data":"2e3bc7abf512b8815dbb25c41b92caed41386b21569b17087b0d4d33cae70a04"}
Jul 06 23:59:04.319232 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:04.318923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"91e0c8854f570d9cae2dc7ed681dec9ce1e8a05a1726780f9b13fea55795d51d"}
Jul 06 23:59:04.557539 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:04.557516    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b78d332e-fe4b-4d9b-bc1f-560995f1cfd5" path="/var/lib/kubelet/pods/b78d332e-fe4b-4d9b-bc1f-560995f1cfd5/volumes"
Jul 06 23:59:04.971782 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:04.971751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:05.032753 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.032722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:05.326657 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"bc4e9fcd8082a9c68549b5296e6bdd0f3b7c7f097c7cb564edf9ce009cbc1480"}
Jul 06 23:59:05.327029 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"e2b8e58e1bb20d0a1eb65d8b5e509c5c7d350d18344f60c261e09c0d54395721"}
Jul 06 23:59:05.327029 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"eca74d7ccde87e4f0741c8e56fae34613a67b60671ad0abb7ba7387debcba5f8"}
Jul 06 23:59:05.327029 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326721    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"dfb42c607bb8928b11030949c75080dd0e1b3d95402c9b62896a6793c11c8cc7"}
Jul 06 23:59:05.327029 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326743    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"0283751639a7471fb3e5dfc7256171375ee6e14788e33fa334441b87845dadcc"}
Jul 06 23:59:05.327029 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.326766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5bf10432-8644-4a6c-a0ef-3708bee2e33a","Type":"ContainerStarted","Data":"c6c06e3790f8cb53bfe98964b0fa6d3e57cee47b3ddd4c1648cfce3fe0db1854"}
Jul 06 23:59:05.360592 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:05.360536    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.343714922 podStartE2EDuration="2.360515175s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.800253ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:05.358466255 +0000 UTC m=+147.761862328" watchObservedRunningTime="2026-07-06 23:59:05.360515175 +0000 UTC m=+147.763911249"
Jul 06 23:59:08.691515 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:08.691463    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 23:59:11.099711 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:11.099680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:11.198459 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:11.198426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:11.556755 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:11.556720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:11.610101 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:11.610075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:11.667868 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:11.667837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:42.144455 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:42.144417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:42.242664 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:42.242638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:56.517345 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:56.517306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:56.617995 ip-10-0-129-135 kubenswrapper[2581]: I0706 23:59:56.617940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:03.691877 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:03.691815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 00:00:03.707096 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:03.707067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 00:00:04.058752 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:04.058701    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:04.582123 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:04.582096    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 07 00:00:05.426503 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:05.426467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:07.659072 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:07.659035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:07.759614 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:07.759574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:12.277579 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:12.277544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:12.377558 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:12.377519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:18.559867 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:18.559832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:18.695376 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:18.695345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:19.193170 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:19.193137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:19.386335 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:19.386301    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:20.792585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:20.792557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:20.991542 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:20.991508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:21.256767 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.256716    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"]
Jul 07 00:00:21.258926 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.258906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.261705 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.261681    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 00:00:21.261820 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.261714    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 00:00:21.262668 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.262651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pdxtq\""
Jul 07 00:00:21.270358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.270337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"]
Jul 07 00:00:21.342001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.341971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.342001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.342005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdrq\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-kube-api-access-xtdrq\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.442531 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.442475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.442531 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.442532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdrq\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-kube-api-access-xtdrq\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.451058 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.451030    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.451219 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.451091    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdrq\" (UniqueName: \"kubernetes.io/projected/df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8-kube-api-access-xtdrq\") pod \"cert-manager-cainjector-559cdd8c87-f7hw7\" (UID: \"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.601595 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.601572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"
Jul 07 00:00:21.731118 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:21.731085    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7"]
Jul 07 00:00:21.733781 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:00:21.733757    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf5e3e51_b1ee_4180_8be4_5af2e4a0d9c8.slice/crio-03bd1f831b3e1fba522f3caff1ba6dd8dbe4964e914537e7a70415c8ac00d46c WatchSource:0}: Error finding container 03bd1f831b3e1fba522f3caff1ba6dd8dbe4964e914537e7a70415c8ac00d46c: Status 404 returned error can't find the container with id 03bd1f831b3e1fba522f3caff1ba6dd8dbe4964e914537e7a70415c8ac00d46c
Jul 07 00:00:22.640209 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:22.640168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7" event={"ID":"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8","Type":"ContainerStarted","Data":"03bd1f831b3e1fba522f3caff1ba6dd8dbe4964e914537e7a70415c8ac00d46c"}
Jul 07 00:00:25.357574 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.357536    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jnhvz"]
Jul 07 00:00:25.360910 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.360891    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.363592 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.363571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-st52d\""
Jul 07 00:00:25.370722 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.370699    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jnhvz"]
Jul 07 00:00:25.372091 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.372070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.372203 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.372097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s4pf\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-kube-api-access-7s4pf\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.472453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.472420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.472453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.472452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s4pf\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-kube-api-access-7s4pf\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.481249 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.481226    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.481463 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.481444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s4pf\" (UniqueName: \"kubernetes.io/projected/420d05e8-b392-4e75-bc8f-c4f652b1a2bb-kube-api-access-7s4pf\") pod \"cert-manager-webhook-6486ffd997-jnhvz\" (UID: \"420d05e8-b392-4e75-bc8f-c4f652b1a2bb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.657187 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.657105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7" event={"ID":"df5e3e51-b1ee-4180-8be4-5af2e4a0d9c8","Type":"ContainerStarted","Data":"4c275fd6efb0695da4cafecdafed0f241020026400fbb9f664e3e22a0a95c215"}
Jul 07 00:00:25.670454 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.670426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:25.674090 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.674046    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f7hw7" podStartSLOduration=1.502893008 podStartE2EDuration="4.674031207s" totalImagesPullingTime="3.171138199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:25.672553801 +0000 UTC m=+228.075949899" watchObservedRunningTime="2026-07-07 00:00:25.674031207 +0000 UTC m=+228.077427279"
Jul 07 00:00:25.803556 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:25.803533    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jnhvz"]
Jul 07 00:00:25.805714 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:00:25.805687    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod420d05e8_b392_4e75_bc8f_c4f652b1a2bb.slice/crio-d9b63f574c05ea6f5bece495388f8b384eaf0ecb59b63937977c725cf4e340b3 WatchSource:0}: Error finding container d9b63f574c05ea6f5bece495388f8b384eaf0ecb59b63937977c725cf4e340b3: Status 404 returned error can't find the container with id d9b63f574c05ea6f5bece495388f8b384eaf0ecb59b63937977c725cf4e340b3
Jul 07 00:00:26.046080 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.046047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:26.102631 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.102597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:26.663148 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.663108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz" event={"ID":"420d05e8-b392-4e75-bc8f-c4f652b1a2bb","Type":"ContainerStarted","Data":"d64c810ffa1bdf2de7cb0a2ba162921d6e0308716b9e7d4a3d5ebe807af00878"}
Jul 07 00:00:26.663576 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.663160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz" event={"ID":"420d05e8-b392-4e75-bc8f-c4f652b1a2bb","Type":"ContainerStarted","Data":"d9b63f574c05ea6f5bece495388f8b384eaf0ecb59b63937977c725cf4e340b3"}
Jul 07 00:00:26.663576 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.663396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:26.678985 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:26.678935    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz" podStartSLOduration=1.678921643 podStartE2EDuration="1.678921643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:26.677299426 +0000 UTC m=+229.080695493" watchObservedRunningTime="2026-07-07 00:00:26.678921643 +0000 UTC m=+229.082317716"
Jul 07 00:00:29.259525 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:29.259486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:32.669529 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:32.669498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jnhvz"
Jul 07 00:00:54.749125 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:54.749092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:54.845944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:00:54.845908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:08.680122 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:08.680045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:08.768344 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:08.768307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:10.759920 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:10.759882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:14.935395 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.935354    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:14.972683 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.972649    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"]
Jul 07 00:01:14.974974 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.974955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:14.978863 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.978831    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 00:01:14.978986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.978913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 00:01:14.978986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.978942    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 00:01:14.979103 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.979005    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 00:01:14.979166 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.979143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4s9zb\""
Jul 07 00:01:14.979248 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.979226    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 00:01:14.992297 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:14.992271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"]
Jul 07 00:01:15.000547 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.000522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/400b3e77-a624-4844-9f5d-8c658a9fa003-manager-config\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.000649 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.000561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.000649 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.000584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-metrics-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.000728 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.000645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgcnd\" (UniqueName: \"kubernetes.io/projected/400b3e77-a624-4844-9f5d-8c658a9fa003-kube-api-access-qgcnd\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.039827 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.039784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:15.102599 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.102564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/400b3e77-a624-4844-9f5d-8c658a9fa003-manager-config\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.102784 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.102620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.102784 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.102652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-metrics-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.102784 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.102683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgcnd\" (UniqueName: \"kubernetes.io/projected/400b3e77-a624-4844-9f5d-8c658a9fa003-kube-api-access-qgcnd\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.103392 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.103364    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/400b3e77-a624-4844-9f5d-8c658a9fa003-manager-config\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.106370 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.106344    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-metrics-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.106493 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.106356    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400b3e77-a624-4844-9f5d-8c658a9fa003-cert\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.118216 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.118195    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgcnd\" (UniqueName: \"kubernetes.io/projected/400b3e77-a624-4844-9f5d-8c658a9fa003-kube-api-access-qgcnd\") pod \"lws-controller-manager-5c45554c7d-kbbxp\" (UID: \"400b3e77-a624-4844-9f5d-8c658a9fa003\") " pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.284235 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.284196    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:15.425959 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.425915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"]
Jul 07 00:01:15.428733 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:01:15.428705    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400b3e77_a624_4844_9f5d_8c658a9fa003.slice/crio-87cb3415424cecee4f8dd206bd834ce67e87bdaef959e2e320601d0b0b4d6609 WatchSource:0}: Error finding container 87cb3415424cecee4f8dd206bd834ce67e87bdaef959e2e320601d0b0b4d6609: Status 404 returned error can't find the container with id 87cb3415424cecee4f8dd206bd834ce67e87bdaef959e2e320601d0b0b4d6609
Jul 07 00:01:15.855101 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:15.855068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp" event={"ID":"400b3e77-a624-4844-9f5d-8c658a9fa003","Type":"ContainerStarted","Data":"87cb3415424cecee4f8dd206bd834ce67e87bdaef959e2e320601d0b0b4d6609"}
Jul 07 00:01:17.864693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:17.864599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp" event={"ID":"400b3e77-a624-4844-9f5d-8c658a9fa003","Type":"ContainerStarted","Data":"df9e7b761dfe87f1eb3b53a5852f45ae344b9309aaba1d6bcc3988f600eb5d98"}
Jul 07 00:01:17.864693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:17.864668    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:17.887107 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:17.887050    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp" podStartSLOduration=1.742770449 podStartE2EDuration="3.887033039s" totalImagesPullingTime="2.14426259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:17.885844891 +0000 UTC m=+280.289240989" watchObservedRunningTime="2026-07-07 00:01:17.887033039 +0000 UTC m=+280.290429175"
Jul 07 00:01:25.310319 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:25.310285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:25.488139 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:25.488092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:25.941347 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:25.941307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:26.585911 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.585876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:26.701035 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.701003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:26.781256 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.781225    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"]
Jul 07 00:01:26.784945 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.784922    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:26.787977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.787952    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jl66k\""
Jul 07 00:01:26.788220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.788195    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 00:01:26.788345 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.788302    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 00:01:26.788409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.788317    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 00:01:26.788617 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.788602    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 00:01:26.801243 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.801222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"]
Jul 07 00:01:26.907012 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.906931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:26.907012 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.906987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxnnw\" (UniqueName: \"kubernetes.io/projected/454c713c-b8cf-4016-8318-7d292e93a31a-kube-api-access-bxnnw\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:26.907211 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.907056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:26.907211 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.907081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:26.907211 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:26.907109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.007715 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.007683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.007715 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.007717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.007945 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.007745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.007945 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.007770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.007945 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.007820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxnnw\" (UniqueName: \"kubernetes.io/projected/454c713c-b8cf-4016-8318-7d292e93a31a-kube-api-access-bxnnw\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.008229 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.008210    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.010074 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.010049    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/454c713c-b8cf-4016-8318-7d292e93a31a-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.010328 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.010306    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.010408 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.010380    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/454c713c-b8cf-4016-8318-7d292e93a31a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.019095 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.019073    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxnnw\" (UniqueName: \"kubernetes.io/projected/454c713c-b8cf-4016-8318-7d292e93a31a-kube-api-access-bxnnw\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-8tdd7\" (UID: \"454c713c-b8cf-4016-8318-7d292e93a31a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.048738 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.048707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.095209 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.095175    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:27.154190 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.154152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.242517 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.242482    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.244066 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.244046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"]
Jul 07 00:01:27.246431 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:01:27.246407    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454c713c_b8cf_4016_8318_7d292e93a31a.slice/crio-50c4f6cd52e93802144a6b56111029906c7e5ed56a4d01a3a7a838dcee8f2083 WatchSource:0}: Error finding container 50c4f6cd52e93802144a6b56111029906c7e5ed56a4d01a3a7a838dcee8f2083: Status 404 returned error can't find the container with id 50c4f6cd52e93802144a6b56111029906c7e5ed56a4d01a3a7a838dcee8f2083
Jul 07 00:01:27.315918 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.315883    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.421456 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.421424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.588056 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.588018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.739424 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.739382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:27.914361 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:27.914272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7" event={"ID":"454c713c-b8cf-4016-8318-7d292e93a31a","Type":"ContainerStarted","Data":"50c4f6cd52e93802144a6b56111029906c7e5ed56a4d01a3a7a838dcee8f2083"}
Jul 07 00:01:28.876509 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:28.876477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c45554c7d-kbbxp"
Jul 07 00:01:30.931100 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:30.931062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7" event={"ID":"454c713c-b8cf-4016-8318-7d292e93a31a","Type":"ContainerStarted","Data":"5688bbaf2a96963990ee575eb5e816d4802eadaf0041b574aef2ee28eefd1981"}
Jul 07 00:01:31.937016 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:31.936933    2581 generic.go:356] "Generic (PLEG): container finished" podID="454c713c-b8cf-4016-8318-7d292e93a31a" containerID="5688bbaf2a96963990ee575eb5e816d4802eadaf0041b574aef2ee28eefd1981" exitCode=0
Jul 07 00:01:31.937016 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:31.937002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7" event={"ID":"454c713c-b8cf-4016-8318-7d292e93a31a","Type":"ContainerDied","Data":"5688bbaf2a96963990ee575eb5e816d4802eadaf0041b574aef2ee28eefd1981"}
Jul 07 00:01:32.943189 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:32.943146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7" event={"ID":"454c713c-b8cf-4016-8318-7d292e93a31a","Type":"ContainerStarted","Data":"2b5a53a208dcfcc39612c487de51f952d44969070351bc8d4f708f953c92e5ac"}
Jul 07 00:01:32.943569 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:32.943289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:33.011877 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:33.011815    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7" podStartSLOduration=2.773561747 podStartE2EDuration="7.011798997s" totalImagesPullingTime="3.174669146s" totalInitContainerRuntime="1.063568104s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:33.006275413 +0000 UTC m=+295.409671497" watchObservedRunningTime="2026-07-07 00:01:33.011798997 +0000 UTC m=+295.415195069"
Jul 07 00:01:38.430392 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:38.430360    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 00:01:43.949938 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:43.949904    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-8tdd7"
Jul 07 00:01:46.668189 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:46.668159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:46.770705 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:46.770667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.114287 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.114251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.306176 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.306144    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"]
Jul 07 00:01:47.308805 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.308782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.316079 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.316035    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 00:01:47.316219 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.316198    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 00:01:47.317305 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.317284    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t75rm\""
Jul 07 00:01:47.317446 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.317315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 00:01:47.317446 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.317318    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 00:01:47.338950 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.338921    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"]
Jul 07 00:01:47.379136 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.379060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.489667 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.489633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tls-certs\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.489667 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.489665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8znk\" (UniqueName: \"kubernetes.io/projected/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-kube-api-access-k8znk\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.489900 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.489696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tmp\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.566836 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.566797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.590399 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.590365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tls-certs\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.590588 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.590406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8znk\" (UniqueName: \"kubernetes.io/projected/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-kube-api-access-k8znk\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.590588 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.590536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tmp\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.593060 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.593032    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tls-certs\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.593207 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.593079    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-tmp\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.599419 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.599385    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8znk\" (UniqueName: \"kubernetes.io/projected/2b6c3f8e-6c09-4d6b-8b32-3df74058ede5-kube-api-access-k8znk\") pod \"kube-auth-proxy-56d7555db7-rh64d\" (UID: \"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.618380 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.618340    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"
Jul 07 00:01:47.670084 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.670042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.756677 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.756643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:47.781895 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.781650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-rh64d"]
Jul 07 00:01:47.800758 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:01:47.800712    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6c3f8e_6c09_4d6b_8b32_3df74058ede5.slice/crio-4ec83a34fa0bcc0482652dbcb90bc545b07252c39b062c94e7de67df74922803 WatchSource:0}: Error finding container 4ec83a34fa0bcc0482652dbcb90bc545b07252c39b062c94e7de67df74922803: Status 404 returned error can't find the container with id 4ec83a34fa0bcc0482652dbcb90bc545b07252c39b062c94e7de67df74922803
Jul 07 00:01:47.804930 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:47.803809    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:01:48.013333 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:48.013298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d" event={"ID":"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5","Type":"ContainerStarted","Data":"4ec83a34fa0bcc0482652dbcb90bc545b07252c39b062c94e7de67df74922803"}
Jul 07 00:01:48.046982 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:48.046948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:48.453621 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:48.453529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:48.689068 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:48.689032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:48.976720 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:48.976686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:49.061693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:49.061645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:49.951616 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:49.951578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:51.028554 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:51.028518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d" event={"ID":"2b6c3f8e-6c09-4d6b-8b32-3df74058ede5","Type":"ContainerStarted","Data":"dc21747350f492514cb7950520c415a31532748e5aa0b99f962fd1d7644a44dd"}
Jul 07 00:01:51.045796 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:51.045745    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56d7555db7-rh64d" podStartSLOduration=1.840760342 podStartE2EDuration="4.045728199s" totalImagesPullingTime="2.204967857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:51.044134029 +0000 UTC m=+313.447530128" watchObservedRunningTime="2026-07-07 00:01:51.045728199 +0000 UTC m=+313.449124441"
Jul 07 00:01:54.856037 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:54.856000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:54.955127 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:54.955086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:55.156224 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:55.156145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:55.352068 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:55.352032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:55.461649 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:55.461568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:55.930250 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:55.930217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:57.734732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:57.734703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:57.872770 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:57.872732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:58.440739 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:58.440694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:58.943567 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:58.943525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:59.099470 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:59.099436    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:59.200606 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:59.200519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:59.308288 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:01:59.308254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:14.295823 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:14.295789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:48.193638 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:48.193602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:48.289805 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:48.289773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:48.429795 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:48.429756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:49.155405 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:49.155373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:49.342324 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:02:49.342288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:12.277471 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:12.277438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:15.667342 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:15.667305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:16.014288 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:16.014257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:16.790842 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:16.790800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:16.885477 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:16.885417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:20.274231 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:20.274190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:20.880429 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:20.880390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:22.670353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:22.670324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:24.139352 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:24.139320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:24.917898 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:24.917847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:25.029314 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:25.029260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:26.470738 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.470703    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"]
Jul 07 00:03:26.473617 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.473601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:26.476277 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:03:26.476246    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-operator-controller-manager-864dd78c4c-6jlnh\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="2dac88b5-a509-4c69-9da5-f9aced210220" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:26.476431 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:03:26.476403    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 07 00:03:26.476488 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:03:26.476403    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-operator-controller-env\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" type="*v1.ConfigMap"
Jul 07 00:03:26.476488 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:03:26.476404    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-9p72x\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9p72x\"" type="*v1.Secret"
Jul 07 00:03:26.486918 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.486902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 00:03:26.500913 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.500890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxlr\" (UniqueName: \"kubernetes.io/projected/2dac88b5-a509-4c69-9da5-f9aced210220-kube-api-access-7cxlr\") pod \"dns-operator-controller-manager-864dd78c4c-6jlnh\" (UID: \"2dac88b5-a509-4c69-9da5-f9aced210220\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:26.508471 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.508446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"]
Jul 07 00:03:26.602287 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.602250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxlr\" (UniqueName: \"kubernetes.io/projected/2dac88b5-a509-4c69-9da5-f9aced210220-kube-api-access-7cxlr\") pod \"dns-operator-controller-manager-864dd78c4c-6jlnh\" (UID: \"2dac88b5-a509-4c69-9da5-f9aced210220\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:26.916781 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:26.916701    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:27.458065 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.458035    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 00:03:27.464712 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.464683    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxlr\" (UniqueName: \"kubernetes.io/projected/2dac88b5-a509-4c69-9da5-f9aced210220-kube-api-access-7cxlr\") pod \"dns-operator-controller-manager-864dd78c4c-6jlnh\" (UID: \"2dac88b5-a509-4c69-9da5-f9aced210220\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:27.529404 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.529375    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 00:03:27.750664 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.750632    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9p72x\""
Jul 07 00:03:27.754067 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.754046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:27.830935 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.830901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:27.892221 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.892195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"]
Jul 07 00:03:27.894024 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:03:27.893998    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dac88b5_a509_4c69_9da5_f9aced210220.slice/crio-31c125abfaca3fd1a7075f2a35b2e348d16a577e9fd28e1c782765f806557daa WatchSource:0}: Error finding container 31c125abfaca3fd1a7075f2a35b2e348d16a577e9fd28e1c782765f806557daa: Status 404 returned error can't find the container with id 31c125abfaca3fd1a7075f2a35b2e348d16a577e9fd28e1c782765f806557daa
Jul 07 00:03:27.941398 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:27.941365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.048193 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.048112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.404331 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.404261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.465591 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.465558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh" event={"ID":"2dac88b5-a509-4c69-9da5-f9aced210220","Type":"ContainerStarted","Data":"31c125abfaca3fd1a7075f2a35b2e348d16a577e9fd28e1c782765f806557daa"}
Jul 07 00:03:28.535560 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.535528    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"]
Jul 07 00:03:28.538104 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.538082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.542908 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.542359    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 00:03:28.542908 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.542613    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 00:03:28.542908 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.542887    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4nstc\""
Jul 07 00:03:28.605165 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.605132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.615322 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.615296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"]
Jul 07 00:03:28.619944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.619917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4846c3e9-9731-4323-a74c-79208577f305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.620079 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.619989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n2dj\" (UniqueName: \"kubernetes.io/projected/4846c3e9-9731-4323-a74c-79208577f305-kube-api-access-7n2dj\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.620161 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.620066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4846c3e9-9731-4323-a74c-79208577f305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.721247 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.721218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4846c3e9-9731-4323-a74c-79208577f305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.721380 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.721312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n2dj\" (UniqueName: \"kubernetes.io/projected/4846c3e9-9731-4323-a74c-79208577f305-kube-api-access-7n2dj\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.721380 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.721347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4846c3e9-9731-4323-a74c-79208577f305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.722013 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.721991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4846c3e9-9731-4323-a74c-79208577f305-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.723554 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.723527    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4846c3e9-9731-4323-a74c-79208577f305-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.740715 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.740671    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n2dj\" (UniqueName: \"kubernetes.io/projected/4846c3e9-9731-4323-a74c-79208577f305-kube-api-access-7n2dj\") pod \"kuadrant-console-plugin-66b5774985-z4drd\" (UID: \"4846c3e9-9731-4323-a74c-79208577f305\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.749571 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.749528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.835437 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.835402    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.850524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.850501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"
Jul 07 00:03:28.995421 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:28.995327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd"]
Jul 07 00:03:28.998279 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:03:28.998246    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4846c3e9_9731_4323_a74c_79208577f305.slice/crio-df782f378b08c7d3c01721a2ae1b94d5fb44d9ce53100ec8aa5e84bb0fece9c1 WatchSource:0}: Error finding container df782f378b08c7d3c01721a2ae1b94d5fb44d9ce53100ec8aa5e84bb0fece9c1: Status 404 returned error can't find the container with id df782f378b08c7d3c01721a2ae1b94d5fb44d9ce53100ec8aa5e84bb0fece9c1
Jul 07 00:03:29.470793 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:29.470758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd" event={"ID":"4846c3e9-9731-4323-a74c-79208577f305","Type":"ContainerStarted","Data":"df782f378b08c7d3c01721a2ae1b94d5fb44d9ce53100ec8aa5e84bb0fece9c1"}
Jul 07 00:03:30.477835 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:30.477795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh" event={"ID":"2dac88b5-a509-4c69-9da5-f9aced210220","Type":"ContainerStarted","Data":"f39536cfd031be81ad5c494ee35370d35e09c7e4e16f81d1545dccd04fcce0fd"}
Jul 07 00:03:30.478310 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:30.477902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:30.511548 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:30.511448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh" podStartSLOduration=2.128811753 podStartE2EDuration="4.51142748s" totalImagesPullingTime="2.382615727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:30.510272905 +0000 UTC m=+412.913669006" watchObservedRunningTime="2026-07-07 00:03:30.51142748 +0000 UTC m=+412.914823558"
Jul 07 00:03:31.873422 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:31.873369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:31.977638 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:31.977595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:40.432031 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:40.431991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:40.608282 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:40.608249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:41.154552 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.154517    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:03:41.188015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.187983    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:03:41.188197 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.188107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.192986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.192956    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9glnk\""
Jul 07 00:03:41.250263 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.250225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.250454 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.250378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6l5x\" (UniqueName: \"kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.351271 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.351230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.351463 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.351329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6l5x\" (UniqueName: \"kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.351655 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.351631    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.363843 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.363813    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6l5x\" (UniqueName: \"kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x\") pod \"kuadrant-operator-controller-manager-86878c769f-wk68v\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:41.485055 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.485021    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-6jlnh"
Jul 07 00:03:41.501985 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:41.501946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:42.828008 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:42.827972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.949303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:42.949273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:46.994722 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:46.994679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:47.120545 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:47.120510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:47.725419 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:47.725376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:47.816659 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:47.816627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:49.260506 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:49.260347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:49.360599 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:49.360564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:49.943465 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:49.943443    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:03:50.070527 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:03:50.070451    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77aadc3f_8697_4d97_8727_bfe55d3e00d0.slice/crio-afef61d11e468ffb3220073d5a14ed50f7959e3e076d28ff11587992a6fb40ec WatchSource:0}: Error finding container afef61d11e468ffb3220073d5a14ed50f7959e3e076d28ff11587992a6fb40ec: Status 404 returned error can't find the container with id afef61d11e468ffb3220073d5a14ed50f7959e3e076d28ff11587992a6fb40ec
Jul 07 00:03:50.585951 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:50.585913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" event={"ID":"77aadc3f-8697-4d97-8727-bfe55d3e00d0","Type":"ContainerStarted","Data":"afef61d11e468ffb3220073d5a14ed50f7959e3e076d28ff11587992a6fb40ec"}
Jul 07 00:03:50.587431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:50.587376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd" event={"ID":"4846c3e9-9731-4323-a74c-79208577f305","Type":"ContainerStarted","Data":"9c1d40681e8538657e7181cc374efe3f262cf9971e5d7b1b9b8a70fe910565f5"}
Jul 07 00:03:50.607212 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:50.607165    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4drd" podStartSLOduration=1.488366644 podStartE2EDuration="22.607150326s" totalImagesPullingTime="21.118783682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:50.60662939 +0000 UTC m=+433.010025461" watchObservedRunningTime="2026-07-07 00:03:50.607150326 +0000 UTC m=+433.010546399"
Jul 07 00:03:58.016564 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:58.016517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:58.151035 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:58.151004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:58.624001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:58.623962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" event={"ID":"77aadc3f-8697-4d97-8727-bfe55d3e00d0","Type":"ContainerStarted","Data":"406cadb37e4e5b8ec309bc67d54a6f08d285df008d92208ddc2dd801f3e813b7"}
Jul 07 00:03:58.624181 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:58.624101    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:03:58.655880 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:03:58.655804    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" podStartSLOduration=10.122676026 podStartE2EDuration="17.655789753s" totalImagesPullingTime="7.533113727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:58.653778017 +0000 UTC m=+441.057174091" watchObservedRunningTime="2026-07-07 00:03:58.655789753 +0000 UTC m=+441.059185826"
Jul 07 00:04:00.470486 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:00.470407    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:00.576152 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:00.576118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:05.895749 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:05.895714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:06.018654 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:06.018619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:07.407666 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:07.407639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:07.509063 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:07.509031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:08.427252 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:08.427206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:08.527075 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:08.527047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:09.630872 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:09.630828    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:04:15.382164 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:15.382129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:17.449412 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:17.449379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:17.550659 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:17.550624    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.064834 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:19.064805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.190124 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:19.190091    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.814527 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:19.814495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.910978 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:19.910947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:26.207303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:26.207267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:26.417936 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:26.417905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:28.333457 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:28.333425    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:28.440839 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:28.440805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:28.509469 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:28.509439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:29.200123 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:29.200084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:29.302656 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:29.302620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:30.432144 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:30.432074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:30.492428 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:30.492391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:44.921286 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:44.921250    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:56.827567 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:56.827529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:57.179468 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:04:57.179384    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:03.894850 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:03.894815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:03.992303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:03.992271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:04.051029 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:04.050995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:04.131189 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:04.131140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:07.531477 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:07.531440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:07.617034 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:07.617002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:39.072993 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:39.072961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:39.543055 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:39.543023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:40.020193 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.020157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:40.120103 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.120066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:40.155663 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.155630    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t7tvs"]
Jul 07 00:05:40.157951 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.157934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.161007 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.160986    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 00:05:40.175915 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.175893    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t7tvs"]
Jul 07 00:05:40.205236 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.205205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.205377 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.205312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjjdx\" (UniqueName: \"kubernetes.io/projected/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-kube-api-access-mjjdx\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.235905 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.235872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:40.306498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.306433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.306498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.306494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjjdx\" (UniqueName: \"kubernetes.io/projected/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-kube-api-access-mjjdx\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.307143 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.307124    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.315832 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.315810    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjjdx\" (UniqueName: \"kubernetes.io/projected/9208b2e8-54bc-4319-8b94-fd9f6d3bb37c-kube-api-access-mjjdx\") pod \"payload-pre-processing-7964457696-t7tvs\" (UID: \"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c\") " pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.468423 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.468385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:40.606719 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:40.606634    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t7tvs"]
Jul 07 00:05:40.611149 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:05:40.611120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9208b2e8_54bc_4319_8b94_fd9f6d3bb37c.slice/crio-59cb7c4c31323b79e15089894b71679094e52a3d0352d1e7437c754a8f8a04d1 WatchSource:0}: Error finding container 59cb7c4c31323b79e15089894b71679094e52a3d0352d1e7437c754a8f8a04d1: Status 404 returned error can't find the container with id 59cb7c4c31323b79e15089894b71679094e52a3d0352d1e7437c754a8f8a04d1
Jul 07 00:05:41.098231 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:41.098197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs" event={"ID":"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c","Type":"ContainerStarted","Data":"59cb7c4c31323b79e15089894b71679094e52a3d0352d1e7437c754a8f8a04d1"}
Jul 07 00:05:42.104000 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:42.103965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs" event={"ID":"9208b2e8-54bc-4319-8b94-fd9f6d3bb37c","Type":"ContainerStarted","Data":"742a6c3782160cc51b801749abccad515f5dcb6afc06699be609927f99188dfb"}
Jul 07 00:05:42.104381 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:42.104080    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:42.129241 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:42.129148    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs" podStartSLOduration=0.866508129 podStartE2EDuration="2.129132615s" totalImagesPullingTime="1.262624486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:42.128301234 +0000 UTC m=+544.531697310" watchObservedRunningTime="2026-07-07 00:05:42.129132615 +0000 UTC m=+544.532528688"
Jul 07 00:05:50.895005 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:50.894962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:51.054610 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:51.054572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:53.110409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:53.110378    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-t7tvs"
Jul 07 00:05:58.757524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:05:58.757475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:18.876920 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.876883    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"]
Jul 07 00:06:18.882009 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.881986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:18.886544 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.886525    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 00:06:18.886642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.886593    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 00:06:18.887775 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.887749    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bvnll\""
Jul 07 00:06:18.887908 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.887814    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 00:06:18.896296 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.896274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"]
Jul 07 00:06:18.928184 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:18.928147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:19.051273 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.051273 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5wbg\" (UniqueName: \"kubernetes.io/projected/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-kube-api-access-k5wbg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.051498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.051498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.051498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.051646 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.051510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152666 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152666 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152666 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5wbg\" (UniqueName: \"kubernetes.io/projected/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-kube-api-access-k5wbg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152975 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152975 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.152975 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.152725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.153126 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.153082    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.153176 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.153122    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.153216 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.153176    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.154962 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.154941    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.155229 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.155210    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.161391 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.161368    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5wbg\" (UniqueName: \"kubernetes.io/projected/a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e-kube-api-access-k5wbg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wl8bw\" (UID: \"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.192258 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.192232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:19.327509 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:19.327482    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"]
Jul 07 00:06:19.329104 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:06:19.329075    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3578d5d_465d_4ee0_9f1b_8c3d10e60a2e.slice/crio-851e16b7ea280357d2382be2e09290cf115632de34c33d13a14fc83d5d3d8444 WatchSource:0}: Error finding container 851e16b7ea280357d2382be2e09290cf115632de34c33d13a14fc83d5d3d8444: Status 404 returned error can't find the container with id 851e16b7ea280357d2382be2e09290cf115632de34c33d13a14fc83d5d3d8444
Jul 07 00:06:20.284672 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:20.284634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw" event={"ID":"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e","Type":"ContainerStarted","Data":"851e16b7ea280357d2382be2e09290cf115632de34c33d13a14fc83d5d3d8444"}
Jul 07 00:06:21.291036 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:21.290996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw" event={"ID":"a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e","Type":"ContainerStarted","Data":"0f811349fd129a773dfee2ca184e3a5999f0f300f7c0b60cd8775f74589885ec"}
Jul 07 00:06:21.308708 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:21.308657    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw" podStartSLOduration=2.245307959 podStartE2EDuration="3.308638325s" totalImagesPullingTime="1.063330366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:21.307897138 +0000 UTC m=+583.711293222" watchObservedRunningTime="2026-07-07 00:06:21.308638325 +0000 UTC m=+583.712034398"
Jul 07 00:06:22.407195 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:22.407163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:27.819025 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:27.818990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:29.192810 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:29.192769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:29.193271 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:29.192882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:29.205683 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:29.205655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:29.348602 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:29.348572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wl8bw"
Jul 07 00:06:31.188281 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.188242    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"]
Jul 07 00:06:31.191903 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.191885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.194611 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.194592    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 07 00:06:31.208216 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.208192    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"]
Jul 07 00:06:31.216639 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.216618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:31.265979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.265945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.266149 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.266011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6367df-4d2f-4121-a3bf-def991e0033b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.266149 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.266097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.266149 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.266130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.266291 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.266206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmcrd\" (UniqueName: \"kubernetes.io/projected/0f6367df-4d2f-4121-a3bf-def991e0033b-kube-api-access-wmcrd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.266291 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.266240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367119 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6367df-4d2f-4121-a3bf-def991e0033b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmcrd\" (UniqueName: \"kubernetes.io/projected/0f6367df-4d2f-4121-a3bf-def991e0033b-kube-api-access-wmcrd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367586 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367586 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367549    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.367693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.367597    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.369443 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.369418    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6367df-4d2f-4121-a3bf-def991e0033b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.369690 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.369673    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6367df-4d2f-4121-a3bf-def991e0033b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.376047 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.376016    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmcrd\" (UniqueName: \"kubernetes.io/projected/0f6367df-4d2f-4121-a3bf-def991e0033b-kube-api-access-wmcrd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr\" (UID: \"0f6367df-4d2f-4121-a3bf-def991e0033b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.502635 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.502596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:31.634778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:31.634674    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"]
Jul 07 00:06:31.636769 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:06:31.636746    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6367df_4d2f_4121_a3bf_def991e0033b.slice/crio-af3b1eae85e9702f64b41e13ba8485127f768dd4527294380c34250595785bfd WatchSource:0}: Error finding container af3b1eae85e9702f64b41e13ba8485127f768dd4527294380c34250595785bfd: Status 404 returned error can't find the container with id af3b1eae85e9702f64b41e13ba8485127f768dd4527294380c34250595785bfd
Jul 07 00:06:32.354192 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:32.354160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr" event={"ID":"0f6367df-4d2f-4121-a3bf-def991e0033b","Type":"ContainerStarted","Data":"becd63c2c9456e862928c0173b74b83e4b2c9ff3d814232500259f8c26b7f1fa"}
Jul 07 00:06:32.354577 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:32.354214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr" event={"ID":"0f6367df-4d2f-4121-a3bf-def991e0033b","Type":"ContainerStarted","Data":"af3b1eae85e9702f64b41e13ba8485127f768dd4527294380c34250595785bfd"}
Jul 07 00:06:32.382528 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:32.382479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr" podStartSLOduration=1.209240103 podStartE2EDuration="1.382461792s" totalImagesPullingTime="173.221689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:32.381195568 +0000 UTC m=+594.784591668" watchObservedRunningTime="2026-07-07 00:06:32.382461792 +0000 UTC m=+594.785857864"
Jul 07 00:06:38.788080 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:38.788042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:41.503390 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:41.503361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:41.503899 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:41.503444    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:41.515962 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:41.515939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:42.322149 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:42.322119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:42.416903 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:42.416846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr"
Jul 07 00:06:43.903112 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:43.903076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:44.004094 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:44.004066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:53.770396 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:06:53.770359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:05.666321 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:08:05.666286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:05.737186 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:08:05.737152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:06.875317 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:08:06.875281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:06.969626 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:08:06.969592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:09:27.216395 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:09:27.216356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:09:28.608721 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:09:28.608687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:07.340466 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:07.340380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:07.438178 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:07.438148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:07.578076 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:07.578036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:13.370199 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:13.370163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:13.428324 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:13.428288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:13.508701 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:13.508666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:13.605810 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:13.605780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:13.749962 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:13.749928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:19.293648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:19.293610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:19.393479 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:10:19.393444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:11:29.912870 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:11:29.912773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:11:30.953917 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:11:30.953884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:52.609999 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:12:52.609966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:14:13.047836 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:13.047793    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:14:13.048440 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:13.048060    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" podUID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" containerName="manager" containerID="cri-o://406cadb37e4e5b8ec309bc67d54a6f08d285df008d92208ddc2dd801f3e813b7" gracePeriod=10
Jul 07 00:14:16.611775 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.611737    2581 generic.go:356] "Generic (PLEG): container finished" podID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" containerID="406cadb37e4e5b8ec309bc67d54a6f08d285df008d92208ddc2dd801f3e813b7" exitCode=0
Jul 07 00:14:16.612170 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.611803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" event={"ID":"77aadc3f-8697-4d97-8727-bfe55d3e00d0","Type":"ContainerDied","Data":"406cadb37e4e5b8ec309bc67d54a6f08d285df008d92208ddc2dd801f3e813b7"}
Jul 07 00:14:16.702936 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.702911    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:14:16.794967 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.794934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume\") pod \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") "
Jul 07 00:14:16.795157 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.795044    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6l5x\" (UniqueName: \"kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x\") pod \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\" (UID: \"77aadc3f-8697-4d97-8727-bfe55d3e00d0\") "
Jul 07 00:14:16.795360 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.795337    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "77aadc3f-8697-4d97-8727-bfe55d3e00d0" (UID: "77aadc3f-8697-4d97-8727-bfe55d3e00d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:14:16.797126 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.797100    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x" (OuterVolumeSpecName: "kube-api-access-j6l5x") pod "77aadc3f-8697-4d97-8727-bfe55d3e00d0" (UID: "77aadc3f-8697-4d97-8727-bfe55d3e00d0"). InnerVolumeSpecName "kube-api-access-j6l5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:14:16.896670 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.896632    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6l5x\" (UniqueName: \"kubernetes.io/projected/77aadc3f-8697-4d97-8727-bfe55d3e00d0-kube-api-access-j6l5x\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:14:16.896670 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:16.896661    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77aadc3f-8697-4d97-8727-bfe55d3e00d0-extensions-socket-volume\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:14:17.617986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:17.617947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v" event={"ID":"77aadc3f-8697-4d97-8727-bfe55d3e00d0","Type":"ContainerDied","Data":"afef61d11e468ffb3220073d5a14ed50f7959e3e076d28ff11587992a6fb40ec"}
Jul 07 00:14:17.617986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:17.617967    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"
Jul 07 00:14:17.618502 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:17.618008    2581 scope.go:117] "RemoveContainer" containerID="406cadb37e4e5b8ec309bc67d54a6f08d285df008d92208ddc2dd801f3e813b7"
Jul 07 00:14:17.648315 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:17.648284    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:14:17.652562 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:17.652538    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wk68v"]
Jul 07 00:14:18.558709 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:14:18.558666    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" path="/var/lib/kubelet/pods/77aadc3f-8697-4d97-8727-bfe55d3e00d0/volumes"
Jul 07 00:15:00.147798 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.147758    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-4pjbr"]
Jul 07 00:15:00.148280 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.148176    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" containerName="manager"
Jul 07 00:15:00.148280 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.148188    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" containerName="manager"
Jul 07 00:15:00.148280 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.148242    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="77aadc3f-8697-4d97-8727-bfe55d3e00d0" containerName="manager"
Jul 07 00:15:00.152461 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.152439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:00.159004 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.158982    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g89gr\""
Jul 07 00:15:00.163773 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.163748    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-4pjbr"]
Jul 07 00:15:00.314125 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.314070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ptts\" (UniqueName: \"kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts\") pod \"maas-api-key-cleanup-29723055-4pjbr\" (UID: \"70fb7991-9c90-4023-a749-49412823b15a\") " pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:00.414966 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.414887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ptts\" (UniqueName: \"kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts\") pod \"maas-api-key-cleanup-29723055-4pjbr\" (UID: \"70fb7991-9c90-4023-a749-49412823b15a\") " pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:00.423522 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.423499    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ptts\" (UniqueName: \"kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts\") pod \"maas-api-key-cleanup-29723055-4pjbr\" (UID: \"70fb7991-9c90-4023-a749-49412823b15a\") " pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:00.463511 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.463482    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:00.592202 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.592178    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-4pjbr"]
Jul 07 00:15:00.594175 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:15:00.594148    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fb7991_9c90_4023_a749_49412823b15a.slice/crio-2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b WatchSource:0}: Error finding container 2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b: Status 404 returned error can't find the container with id 2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b
Jul 07 00:15:00.596359 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.596340    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:15:00.823952 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:00.823904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr" event={"ID":"70fb7991-9c90-4023-a749-49412823b15a","Type":"ContainerStarted","Data":"2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b"}
Jul 07 00:15:03.840325 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:03.840229    2581 generic.go:356] "Generic (PLEG): container finished" podID="70fb7991-9c90-4023-a749-49412823b15a" containerID="654eada1ecc65bde394fcc0dd796ff3647ecdb217ed3994c74f0282369047d75" exitCode=0
Jul 07 00:15:03.840325 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:03.840313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr" event={"ID":"70fb7991-9c90-4023-a749-49412823b15a","Type":"ContainerDied","Data":"654eada1ecc65bde394fcc0dd796ff3647ecdb217ed3994c74f0282369047d75"}
Jul 07 00:15:04.980038 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:04.980015    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:05.158633 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.158547    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ptts\" (UniqueName: \"kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts\") pod \"70fb7991-9c90-4023-a749-49412823b15a\" (UID: \"70fb7991-9c90-4023-a749-49412823b15a\") "
Jul 07 00:15:05.160728 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.160706    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts" (OuterVolumeSpecName: "kube-api-access-8ptts") pod "70fb7991-9c90-4023-a749-49412823b15a" (UID: "70fb7991-9c90-4023-a749-49412823b15a"). InnerVolumeSpecName "kube-api-access-8ptts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:15:05.260022 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.259986    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ptts\" (UniqueName: \"kubernetes.io/projected/70fb7991-9c90-4023-a749-49412823b15a-kube-api-access-8ptts\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:15:05.850693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.850660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr" event={"ID":"70fb7991-9c90-4023-a749-49412823b15a","Type":"ContainerDied","Data":"2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b"}
Jul 07 00:15:05.850882 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.850708    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c826c8c48fbe1b13d48937d966346f1320fd60f7ab576b3ad0d991eae949b2b"
Jul 07 00:15:05.850882 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:05.850667    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-4pjbr"
Jul 07 00:15:14.177745 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.177712    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"]
Jul 07 00:15:14.182151 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.179961    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70fb7991-9c90-4023-a749-49412823b15a" containerName="cleanup"
Jul 07 00:15:14.182151 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.179990    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70fb7991-9c90-4023-a749-49412823b15a" containerName="cleanup"
Jul 07 00:15:14.182151 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.180216    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70fb7991-9c90-4023-a749-49412823b15a" containerName="cleanup"
Jul 07 00:15:14.184490 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.184460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.187661 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.187640    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9glnk\""
Jul 07 00:15:14.195455 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.195432    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"]
Jul 07 00:15:14.236387 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.236350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fab5965-cde9-4f7c-b450-9b95afb1ae93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.236387 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.236388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn529\" (UniqueName: \"kubernetes.io/projected/8fab5965-cde9-4f7c-b450-9b95afb1ae93-kube-api-access-mn529\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.336923 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.336884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fab5965-cde9-4f7c-b450-9b95afb1ae93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.337060 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.336933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn529\" (UniqueName: \"kubernetes.io/projected/8fab5965-cde9-4f7c-b450-9b95afb1ae93-kube-api-access-mn529\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.337242 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.337220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fab5965-cde9-4f7c-b450-9b95afb1ae93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.345339 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.345294    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn529\" (UniqueName: \"kubernetes.io/projected/8fab5965-cde9-4f7c-b450-9b95afb1ae93-kube-api-access-mn529\") pod \"kuadrant-operator-controller-manager-86878c769f-n5ph5\" (UID: \"8fab5965-cde9-4f7c-b450-9b95afb1ae93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.495365 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.495329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.635890 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.635789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"]
Jul 07 00:15:14.640599 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:15:14.640564    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fab5965_cde9_4f7c_b450_9b95afb1ae93.slice/crio-3f01e4ca75f36c83ed8f8e2f3a93ea358b319552ed6f63912c3de921c5e1d29d WatchSource:0}: Error finding container 3f01e4ca75f36c83ed8f8e2f3a93ea358b319552ed6f63912c3de921c5e1d29d: Status 404 returned error can't find the container with id 3f01e4ca75f36c83ed8f8e2f3a93ea358b319552ed6f63912c3de921c5e1d29d
Jul 07 00:15:14.893487 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.893385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5" event={"ID":"8fab5965-cde9-4f7c-b450-9b95afb1ae93","Type":"ContainerStarted","Data":"42bd946f05c93483eefa5d3c27f59b6038f0a27cba6ba1b8e57a254b92b65ef6"}
Jul 07 00:15:14.893487 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.893447    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5" event={"ID":"8fab5965-cde9-4f7c-b450-9b95afb1ae93","Type":"ContainerStarted","Data":"3f01e4ca75f36c83ed8f8e2f3a93ea358b319552ed6f63912c3de921c5e1d29d"}
Jul 07 00:15:14.893713 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.893492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:15:14.912143 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:14.912089    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5" podStartSLOduration=0.912072155 podStartE2EDuration="912.072155ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:14.91087862 +0000 UTC m=+1117.314274702" watchObservedRunningTime="2026-07-07 00:15:14.912072155 +0000 UTC m=+1117.315468228"
Jul 07 00:15:20.035298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:20.035266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:15:20.119696 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:20.119658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:15:25.900617 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:15:25.900578    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n5ph5"
Jul 07 00:16:13.892197 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:16:13.892114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:05.054717 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:05.054678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:09.600699 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:09.600670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:17.875499 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:17.875454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:22.974159 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:22.974121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:25.784078 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:25.784026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:25.923794 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:25.923755    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:26.241837 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:26.241794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:32.102953 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:32.102920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:32.203271 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:32.203237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:32.323301 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:32.323264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:32.419807 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:32.419682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:33.441565 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:33.441529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:33.520447 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:33.520406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:33.590397 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:33.590364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:33.690960 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:33.690930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:34.020970 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:34.020934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:34.127464 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:34.127426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:35.812966 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:35.812929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:38.589851 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:38.589816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:38.777513 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:38.777480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:38.885885 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:38.885760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:38.986038 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:38.986002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:41.944112 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:41.944078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:44.910027 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:44.909995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:44.978519 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:44.978476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:45.136504 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:45.136471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:45.237460 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:45.237422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:46.271514 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:46.271476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:46.486878 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:46.486822    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:46.632753 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:46.632651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:46.726815 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:46.726778    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:47.117595 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:47.117560    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:47.211129 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:47.211087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:48.741163 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:48.741124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:51.358532 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:51.358497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:51.718460 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:51.718425    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:57.430497 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:57.430460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:57.531105 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:21:57.531069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:00.090465 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:00.090428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:03.548923 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:03.548887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:03.654724 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:03.654685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:03.825410 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:03.825311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:03.921452 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:03.921414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:04.959081 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:04.959046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:05.059845 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:05.059807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:05.146917 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:05.146886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:05.249231 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:05.249201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:05.649472 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:05.649385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:05.749001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:05.748968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:06.935658 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:06.935626    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:09.342764 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:09.342726    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:09.514696 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:09.514660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:09.619030 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:09.618941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:09.751843 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:09.751813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:10.896997 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:10.896962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:15.600642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:15.600612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:15.701231 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:15.701198    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:16.097872 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:16.097824    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:16.194382 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:16.194345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:16.294975 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:16.294939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:17.663808 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:17.663771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:23.708918 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:23.708885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:23.947069 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:23.947037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:24.076495 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:24.076458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:25.443079 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:25.443043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:29.126819 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:29.126784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:41.364062 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:41.364025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:51.448753 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:51.448707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:54.704683 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:54.704641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:57.340671 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:57.340634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:57.409959 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:57.409928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:57.506579 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:57.506540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:22:57.603625 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:22:57.603535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:13.449344 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:13.449293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:19.306287 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:19.306241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:19.373887 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:19.373827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:19.456045 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:19.456009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:19.556934 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:19.556835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:30.031572 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:30.031478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:30.858917 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:30.858876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:36.662457 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:36.662419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:36.776218 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:36.776175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:36.868078 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:36.868025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:36.968771 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:36.968739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:37.753457 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:37.753398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:37.849526 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:37.849483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:37.949537 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:37.949507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:39.560232 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:39.560192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:45.330228 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:45.330184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:45.431265 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:45.431228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:46.010746 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:46.010709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:51.844654 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:51.844619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:51.908292 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:51.908261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:52.099910 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:52.099813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:52.438474 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:52.438388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:52.552189 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:52.552141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.200086 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.200042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.290266 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.290225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.429007 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.428973    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.741803 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.741776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.891207 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.891171    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:55.992141 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:55.992041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:59.693719 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:23:59.693674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.488007 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:00.487970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.569675 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:00.569638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.862040 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:00.861957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:07.004732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:07.004696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:07.064739 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:07.064706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:07.430588 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:07.430500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:07.822165 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:07.822130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:07.891221 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:07.891187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:13.758736 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:13.758700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:13.858542 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:13.858508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:14.029909 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:14.029802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:14.245215 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:14.245184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:14.366315 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:14.366235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:14.467035 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:14.467000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:14.577961 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:14.577930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.502952 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:20.502912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.573966 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:20.573930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.673448 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:20.673407    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.768114 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:20.768019    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.869729 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:20.869695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:26.513068 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:26.513031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:26.577583 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:26.577546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:33.285330 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:33.285291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:33.608161 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:33.608078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:33.704158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:33.704127    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:36.549195 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:36.549158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:36.650581 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:36.650546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:36.733493 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:36.733439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:37.202999 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:37.202961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:37.303867 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:37.303814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:40.998323 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:40.998290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:46.900326 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.900291    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:24:46.903840 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.903819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:46.906572 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.906548    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0dd24d04\"/\"default-dockercfg-5jpz8\""
Jul 07 00:24:46.906707 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.906548    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0dd24d04\"/\"openshift-service-ca.crt\""
Jul 07 00:24:46.907596 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.907575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0dd24d04\"/\"e2e-lifecycle-model-0dd24d04-kserve-self-signed-certs\""
Jul 07 00:24:46.907687 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.907576    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0dd24d04\"/\"kube-root-ca.crt\""
Jul 07 00:24:46.911979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:46.911960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:24:47.024361 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.024361 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.024566 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5n4v\" (UniqueName: \"kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.024566 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.024566 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.024706 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.024586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.110178 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.110141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:47.125286 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5n4v\" (UniqueName: \"kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125614 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125614 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125841 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125816    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125941 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125844    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.125941 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.125925    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.127663 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.127644    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.127987 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.127963    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.134402 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.134377    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5n4v\" (UniqueName: \"kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v\") pod \"e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") " pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.213992 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.213900    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:47.351267 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.351242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:24:47.353746 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:24:47.353720    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5541c98a_5b2d_4689_8c33_ab2404c49dc4.slice/crio-e51314f4c4211ecf445b553c2e0bc8e0466d74effce7c4ffc43f106d2177e16e WatchSource:0}: Error finding container e51314f4c4211ecf445b553c2e0bc8e0466d74effce7c4ffc43f106d2177e16e: Status 404 returned error can't find the container with id e51314f4c4211ecf445b553c2e0bc8e0466d74effce7c4ffc43f106d2177e16e
Jul 07 00:24:47.730898 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.730841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" event={"ID":"5541c98a-5b2d-4689-8c33-ab2404c49dc4","Type":"ContainerStarted","Data":"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"}
Jul 07 00:24:47.731078 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.730908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" event={"ID":"5541c98a-5b2d-4689-8c33-ab2404c49dc4","Type":"ContainerStarted","Data":"e51314f4c4211ecf445b553c2e0bc8e0466d74effce7c4ffc43f106d2177e16e"}
Jul 07 00:24:47.750197 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:47.750145    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" podStartSLOduration=1.7501276749999999 podStartE2EDuration="1.750127675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:47.748664014 +0000 UTC m=+1690.152060099" watchObservedRunningTime="2026-07-07 00:24:47.750127675 +0000 UTC m=+1690.153523748"
Jul 07 00:24:50.046187 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:50.046156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:50.146021 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:50.145984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:57.214524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:57.214486    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:57.215080 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:57.214542    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:57.226711 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:57.226685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:57.793704 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:57.793677    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:24:57.946134 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:57.946087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:58.048117 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:24:58.048027    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:08.360303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:08.360262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:08.702807 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:08.702720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:14.446002 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:14.445966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:19.425529 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:19.425491    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:25:19.522965 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:19.522928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:33.959428 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:33.959387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:34.446655 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.446614    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" podUID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" containerName="main" containerID="cri-o://ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400" gracePeriod=45
Jul 07 00:25:34.695999 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.695975    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:25:34.868184 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868148    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868381 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868196    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5n4v\" (UniqueName: \"kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868381 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868381 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868286    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868381 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm\") pod \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\" (UID: \"5541c98a-5b2d-4689-8c33-ab2404c49dc4\") "
Jul 07 00:25:34.868585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868439    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache" (OuterVolumeSpecName: "model-cache") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:34.868585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868492    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:34.868585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868542    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home" (OuterVolumeSpecName: "home") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:34.868756 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868738    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.868800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868761    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.868800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.868770    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.870545 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.870515    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:25:34.870545 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.870542    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm" (OuterVolumeSpecName: "dshm") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:25:34.870713 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.870615    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v" (OuterVolumeSpecName: "kube-api-access-n5n4v") pod "5541c98a-5b2d-4689-8c33-ab2404c49dc4" (UID: "5541c98a-5b2d-4689-8c33-ab2404c49dc4"). InnerVolumeSpecName "kube-api-access-n5n4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:25:34.966724 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.966692    2581 generic.go:356] "Generic (PLEG): container finished" podID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" containerID="ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400" exitCode=0
Jul 07 00:25:34.967158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.966734    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"
Jul 07 00:25:34.967158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.966777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" event={"ID":"5541c98a-5b2d-4689-8c33-ab2404c49dc4","Type":"ContainerDied","Data":"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"}
Jul 07 00:25:34.967158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.966819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4" event={"ID":"5541c98a-5b2d-4689-8c33-ab2404c49dc4","Type":"ContainerDied","Data":"e51314f4c4211ecf445b553c2e0bc8e0466d74effce7c4ffc43f106d2177e16e"}
Jul 07 00:25:34.967158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.966836    2581 scope.go:117] "RemoveContainer" containerID="ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"
Jul 07 00:25:34.969621 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.969594    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5n4v\" (UniqueName: \"kubernetes.io/projected/5541c98a-5b2d-4689-8c33-ab2404c49dc4-kube-api-access-n5n4v\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.969621 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.969614    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5541c98a-5b2d-4689-8c33-ab2404c49dc4-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.969621 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.969623    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5541c98a-5b2d-4689-8c33-ab2404c49dc4-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:25:34.975679 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.975665    2581 scope.go:117] "RemoveContainer" containerID="ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"
Jul 07 00:25:34.975965 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:25:34.975947    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400\": container with ID starting with ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400 not found: ID does not exist" containerID="ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"
Jul 07 00:25:34.976046 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.975973    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400"} err="failed to get container status \"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400\": rpc error: code = NotFound desc = could not find container \"ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400\": container with ID starting with ea31f842e1be1307d9b894d3d70d1ec8980c435b9f3994367d7968da3d028400 not found: ID does not exist"
Jul 07 00:25:34.988206 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.988182    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:25:34.991971 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:34.991952    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-0dd24d04/e2e-lifecycle-model-0dd24d04-kserve-5c54bd6ddc-w8mq4"]
Jul 07 00:25:36.557834 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:36.557797    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" path="/var/lib/kubelet/pods/5541c98a-5b2d-4689-8c33-ab2404c49dc4/volumes"
Jul 07 00:25:48.098418 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:48.098385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:48.199270 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:48.199232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:55.574142 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:55.574100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:56.839836 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:25:56.839800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:02.568801 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:02.568769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:02.625459 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:02.625421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:02.838297 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:02.838259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:03.076409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:03.076374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:04.175778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:04.175751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:04.269481 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:04.269449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:04.369914 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:04.369852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:05.163451 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:05.163312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:05.267201 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:05.267163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:05.361528 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:05.361486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:06.861715 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:06.861671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:12.716757 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:12.716717    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:12.820211 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:12.820167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:13.396076 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:13.396042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:19.407390 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:19.407351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:19.474057 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:19.474010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:19.569517 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:19.569479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:19.668596 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:19.668505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:30.147363 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:30.147269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:30.453134 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:30.453038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:51.414114 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:51.414073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:57.732410 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:57.732375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:57.832112 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:57.832076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:58.077429 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:58.077328    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:58.390147 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:58.390056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:58.547093 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:26:58.547054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.030343 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.030302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.122989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.122958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.196800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.196770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.293847 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.293763    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.700444 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.700352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:01.811354 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:01.811316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:05.522846 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:05.522809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:05.869596 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:05.869502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:06.192108 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:06.192029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:06.455327 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:06.455246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.246432 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.246398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.310048 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.310015    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:27:09.310475 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.310457    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" containerName="main"
Jul 07 00:27:09.310475 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.310476    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" containerName="main"
Jul 07 00:27:09.310603 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.310544    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5541c98a-5b2d-4689-8c33-ab2404c49dc4" containerName="main"
Jul 07 00:27:09.313584 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.313561    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.316313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.316287    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g89gr\""
Jul 07 00:27:09.316546 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.316523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-3d04a9-serving-cert\""
Jul 07 00:27:09.324670 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.324647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:27:09.333409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.333384    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.348838 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.348814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zk4v\" (UniqueName: \"kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.348973 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.348894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.348973 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.348928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.416269 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.416232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.449826 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.449789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zk4v\" (UniqueName: \"kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.450033 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.449843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.450033 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.450022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.450653 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.450629    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.452184 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.452165    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.459429 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.459386    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zk4v\" (UniqueName: \"kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v\") pod \"maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") " pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.515329 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.515261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.624822 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.624783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:09.916050 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.915962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.969720 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.969690    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:27:09.972691 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:27:09.972661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d75b299_2a52_4dd3_b5d0_d245152dcc5a.slice/crio-c6c78098995f612261b105c509565c9884e1a2a4c27681794ecc270decf50c1a WatchSource:0}: Error finding container c6c78098995f612261b105c509565c9884e1a2a4c27681794ecc270decf50c1a: Status 404 returned error can't find the container with id c6c78098995f612261b105c509565c9884e1a2a4c27681794ecc270decf50c1a
Jul 07 00:27:09.973965 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.973946    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:27:09.979494 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:09.979471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:10.427081 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:10.427046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" event={"ID":"6d75b299-2a52-4dd3-b5d0-d245152dcc5a","Type":"ContainerStarted","Data":"c6c78098995f612261b105c509565c9884e1a2a4c27681794ecc270decf50c1a"}
Jul 07 00:27:12.439595 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:12.439563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" event={"ID":"6d75b299-2a52-4dd3-b5d0-d245152dcc5a","Type":"ContainerStarted","Data":"21236f75becfb11e285228ecfb790536d3a187b3e28018b1198620c4d1c6dc12"}
Jul 07 00:27:12.440010 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:12.439626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:12.457820 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:12.457769    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" podStartSLOduration=1.553351992 podStartE2EDuration="3.457752437s" totalImagesPullingTime="1.904400445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:12.456723784 +0000 UTC m=+1834.860119883" watchObservedRunningTime="2026-07-07 00:27:12.457752437 +0000 UTC m=+1834.861148510"
Jul 07 00:27:13.735175 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:13.735144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:18.450937 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:18.450907    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:27:22.490720 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:22.490684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:22.591127 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:22.591088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:31.738309 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:31.738271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:31.839347 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:31.839315    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:40.186299 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.186262    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:27:40.190891 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.190850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.193549 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.193527    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7eba6c\"/\"default-dockercfg-6bbbs\""
Jul 07 00:27:40.193648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.193544    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7eba6c\"/\"kube-root-ca.crt\""
Jul 07 00:27:40.194699 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.194684    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7eba6c\"/\"openshift-service-ca.crt\""
Jul 07 00:27:40.194962 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.194938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7eba6c\"/\"test-model-7eba6c-kserve-self-signed-certs\""
Jul 07 00:27:40.198061 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.198042    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:27:40.287182 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.287140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:40.344636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.344813 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344653    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.344813 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.344813 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvtkk\" (UniqueName: \"kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.344948 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.344948 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.344927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446478 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446478 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446478 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446718 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446718 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446718 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvtkk\" (UniqueName: \"kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.446970 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446944    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.447091 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.446997    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.447133 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.447091    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.448750 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.448731    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.448965 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.448949    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.455223 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.455198    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvtkk\" (UniqueName: \"kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk\") pod \"test-model-7eba6c-kserve-7b9bfbb689-zffdz\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") " pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.500693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.500664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:40.635821 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:40.635796    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:27:40.637999 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:27:40.637975    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fe37030_efc2_4ef9_8a9e_a7f3b61f5e5b.slice/crio-86f8b02c00dfe3fc0d98e2cb11f0e079bd86b9e9c93bc62e04278b87fd2c3fa0 WatchSource:0}: Error finding container 86f8b02c00dfe3fc0d98e2cb11f0e079bd86b9e9c93bc62e04278b87fd2c3fa0: Status 404 returned error can't find the container with id 86f8b02c00dfe3fc0d98e2cb11f0e079bd86b9e9c93bc62e04278b87fd2c3fa0
Jul 07 00:27:41.588782 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:41.588748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" event={"ID":"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b","Type":"ContainerStarted","Data":"ed67d7f0e8322db895de5acf59900391ed0ec927762b7be4a025f817a8887187"}
Jul 07 00:27:41.588782 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:41.588788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" event={"ID":"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b","Type":"ContainerStarted","Data":"86f8b02c00dfe3fc0d98e2cb11f0e079bd86b9e9c93bc62e04278b87fd2c3fa0"}
Jul 07 00:27:41.606323 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:41.606275    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" podStartSLOduration=1.6062585660000002 podStartE2EDuration="1.606258566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:41.604767786 +0000 UTC m=+1864.008163863" watchObservedRunningTime="2026-07-07 00:27:41.606258566 +0000 UTC m=+1864.009654639"
Jul 07 00:27:50.500952 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:50.500910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:50.501425 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:50.500985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:50.513494 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:50.513472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:27:50.642998 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:27:50.642970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:28:01.612176 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:01.612086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:22.675569 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:22.675540    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:28:22.746946 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:22.746909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:23.606253 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:23.606218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:23.779064 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:23.779032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:24.191362 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:24.191318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:24.291494 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:24.291456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:24.392643 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:24.392607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:24.496025 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:24.495975    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:29.265403 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:29.265370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:37.695750 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.695687    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" podUID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" containerName="main" containerID="cri-o://ed67d7f0e8322db895de5acf59900391ed0ec927762b7be4a025f817a8887187" gracePeriod=45
Jul 07 00:28:37.876013 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.875984    2581 generic.go:356] "Generic (PLEG): container finished" podID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" containerID="ed67d7f0e8322db895de5acf59900391ed0ec927762b7be4a025f817a8887187" exitCode=0
Jul 07 00:28:37.876162 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.876039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" event={"ID":"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b","Type":"ContainerDied","Data":"ed67d7f0e8322db895de5acf59900391ed0ec927762b7be4a025f817a8887187"}
Jul 07 00:28:37.945568 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.945542    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:28:37.972698 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.972880 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972725    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvtkk\" (UniqueName: \"kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.972880 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972798    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.972880 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972844    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.973134 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.973134 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.972948    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs\") pod \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\" (UID: \"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b\") "
Jul 07 00:28:37.973305 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.973265    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:37.973478 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.973395    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home" (OuterVolumeSpecName: "home") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:37.973653 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.973628    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:37.975572 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.975541    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm" (OuterVolumeSpecName: "dshm") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:37.975665 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.975603    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:28:37.975936 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:37.975921    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk" (OuterVolumeSpecName: "kube-api-access-nvtkk") pod "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" (UID: "6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b"). InnerVolumeSpecName "kube-api-access-nvtkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:28:38.073712 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073678    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.073712 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073703    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.073712 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073712    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvtkk\" (UniqueName: \"kubernetes.io/projected/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-kube-api-access-nvtkk\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.073712 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073721    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.074001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073730    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.074001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.073738    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:28:38.882220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.882125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz" event={"ID":"6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b","Type":"ContainerDied","Data":"86f8b02c00dfe3fc0d98e2cb11f0e079bd86b9e9c93bc62e04278b87fd2c3fa0"}
Jul 07 00:28:38.882220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.882161    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"
Jul 07 00:28:38.882220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.882177    2581 scope.go:117] "RemoveContainer" containerID="ed67d7f0e8322db895de5acf59900391ed0ec927762b7be4a025f817a8887187"
Jul 07 00:28:38.899911 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.899888    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:28:38.901591 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:38.901570    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7eba6c/test-model-7eba6c-kserve-7b9bfbb689-zffdz"]
Jul 07 00:28:40.558113 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:28:40.558072    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" path="/var/lib/kubelet/pods/6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b/volumes"
Jul 07 00:29:07.824511 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:07.824474    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:29:07.825015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:07.824666    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" podUID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" containerName="maas-api" containerID="cri-o://21236f75becfb11e285228ecfb790536d3a187b3e28018b1198620c4d1c6dc12" gracePeriod=30
Jul 07 00:29:08.028706 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.028673    2581 generic.go:356] "Generic (PLEG): container finished" podID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" containerID="21236f75becfb11e285228ecfb790536d3a187b3e28018b1198620c4d1c6dc12" exitCode=0
Jul 07 00:29:08.028886 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.028741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" event={"ID":"6d75b299-2a52-4dd3-b5d0-d245152dcc5a","Type":"ContainerDied","Data":"21236f75becfb11e285228ecfb790536d3a187b3e28018b1198620c4d1c6dc12"}
Jul 07 00:29:08.073580 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.073559    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:29:08.153514 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.153440    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca\") pod \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") "
Jul 07 00:29:08.153514 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.153489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zk4v\" (UniqueName: \"kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v\") pod \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") "
Jul 07 00:29:08.153692 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.153552    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls\") pod \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\" (UID: \"6d75b299-2a52-4dd3-b5d0-d245152dcc5a\") "
Jul 07 00:29:08.153830 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.153808    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d75b299-2a52-4dd3-b5d0-d245152dcc5a" (UID: "6d75b299-2a52-4dd3-b5d0-d245152dcc5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:29:08.153931 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.153919    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-openshift-service-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:08.155511 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.155491    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d75b299-2a52-4dd3-b5d0-d245152dcc5a" (UID: "6d75b299-2a52-4dd3-b5d0-d245152dcc5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:29:08.155562 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.155529    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v" (OuterVolumeSpecName: "kube-api-access-9zk4v") pod "6d75b299-2a52-4dd3-b5d0-d245152dcc5a" (UID: "6d75b299-2a52-4dd3-b5d0-d245152dcc5a"). InnerVolumeSpecName "kube-api-access-9zk4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:08.254846 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.254814    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zk4v\" (UniqueName: \"kubernetes.io/projected/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-kube-api-access-9zk4v\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:08.254846 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:08.254841    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d75b299-2a52-4dd3-b5d0-d245152dcc5a-maas-api-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:09.034256 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:09.034221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx" event={"ID":"6d75b299-2a52-4dd3-b5d0-d245152dcc5a","Type":"ContainerDied","Data":"c6c78098995f612261b105c509565c9884e1a2a4c27681794ecc270decf50c1a"}
Jul 07 00:29:09.034256 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:09.034242    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"
Jul 07 00:29:09.034844 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:09.034281    2581 scope.go:117] "RemoveContainer" containerID="21236f75becfb11e285228ecfb790536d3a187b3e28018b1198620c4d1c6dc12"
Jul 07 00:29:09.050090 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:09.050068    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:29:09.056117 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:09.056090    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3d04a9-5b6b9f7ff6-xbncx"]
Jul 07 00:29:10.558157 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:10.558116    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" path="/var/lib/kubelet/pods/6d75b299-2a52-4dd3-b5d0-d245152dcc5a/volumes"
Jul 07 00:29:17.836360 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.836319    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:17.836961 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.836942    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" containerName="main"
Jul 07 00:29:17.837015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.836965    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" containerName="main"
Jul 07 00:29:17.837015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.836981    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" containerName="maas-api"
Jul 07 00:29:17.837015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.836990    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" containerName="maas-api"
Jul 07 00:29:17.837267 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.837115    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d75b299-2a52-4dd3-b5d0-d245152dcc5a" containerName="maas-api"
Jul 07 00:29:17.837267 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.837134    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fe37030-efc2-4ef9-8a9e-a7f3b61f5e5b" containerName="main"
Jul 07 00:29:17.840691 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.840669    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:17.843063 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.843041    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-2bebf4-serving-cert\""
Jul 07 00:29:17.843184 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.843134    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g89gr\""
Jul 07 00:29:17.849844 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.849820    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:17.947657 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.947624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w45p\" (UniqueName: \"kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:17.947828 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.947671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:17.947828 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:17.947755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.049240 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.049206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w45p\" (UniqueName: \"kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.049410 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.049251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.049410 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.049378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.049500 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:18.049488    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-2bebf4-serving-cert: secret "maas-api-e2e-api-a-2bebf4-serving-cert" not found
Jul 07 00:29:18.049550 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:18.049541    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls podName:fcb48510-adc9-4f07-bd0c-afdbc6371685 nodeName:}" failed. No retries permitted until 2026-07-07 00:29:18.549525924 +0000 UTC m=+1960.952921975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls") pod "maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" (UID: "fcb48510-adc9-4f07-bd0c-afdbc6371685") : secret "maas-api-e2e-api-a-2bebf4-serving-cert" not found
Jul 07 00:29:18.050074 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.050056    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.064369 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.064338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w45p\" (UniqueName: \"kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.554972 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.554939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.557440 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.557415    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") pod \"maas-api-e2e-api-a-2bebf4-8475776f98-98wnc\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") " pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:18.753056 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:18.753022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:19.091435 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:19.091398    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:19.092711 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:29:19.092689    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb48510_adc9_4f07_bd0c_afdbc6371685.slice/crio-3939e39e631aa5d4ce2640f6347532ca81895733d9975eca827f27123d378834 WatchSource:0}: Error finding container 3939e39e631aa5d4ce2640f6347532ca81895733d9975eca827f27123d378834: Status 404 returned error can't find the container with id 3939e39e631aa5d4ce2640f6347532ca81895733d9975eca827f27123d378834
Jul 07 00:29:20.097378 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:20.097340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" event={"ID":"fcb48510-adc9-4f07-bd0c-afdbc6371685","Type":"ContainerStarted","Data":"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"}
Jul 07 00:29:20.097818 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:20.097396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" event={"ID":"fcb48510-adc9-4f07-bd0c-afdbc6371685","Type":"ContainerStarted","Data":"3939e39e631aa5d4ce2640f6347532ca81895733d9975eca827f27123d378834"}
Jul 07 00:29:20.097818 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:20.097442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:20.115943 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:20.115895    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" podStartSLOduration=2.694337736 podStartE2EDuration="3.115882327s" totalImagesPullingTime="421.544591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:20.113798223 +0000 UTC m=+1962.517194297" watchObservedRunningTime="2026-07-07 00:29:20.115882327 +0000 UTC m=+1962.519278397"
Jul 07 00:29:26.097576 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.097280    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:26.101261 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.101238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.104756 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.104726    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-7e3541-serving-cert\""
Jul 07 00:29:26.108699 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.108674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:26.112901 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.112878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:26.225819 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.225787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.225990 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.225973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtmxl\" (UniqueName: \"kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.226054 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.226017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.326740 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.326705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtmxl\" (UniqueName: \"kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.326942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.326754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.326942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.326812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.327028 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:26.326960    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-7e3541-serving-cert: secret "maas-api-e2e-api-b-7e3541-serving-cert" not found
Jul 07 00:29:26.327078 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:26.327031    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls podName:a861cc6a-4cb7-4e66-ada8-051c937c6d0d nodeName:}" failed. No retries permitted until 2026-07-07 00:29:26.827010666 +0000 UTC m=+1969.230406722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls") pod "maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" (UID: "a861cc6a-4cb7-4e66-ada8-051c937c6d0d") : secret "maas-api-e2e-api-b-7e3541-serving-cert" not found
Jul 07 00:29:26.327521 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.327501    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.336043 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.336017    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtmxl\" (UniqueName: \"kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.823177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.823142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:26.831328 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.831299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:26.833580 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:26.833557    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") pod \"maas-api-e2e-api-b-7e3541-66b878c78f-4fk48\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") " pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:27.014932 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:27.014898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:27.145160 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:27.145137    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:27.147063 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:29:27.147032    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda861cc6a_4cb7_4e66_ada8_051c937c6d0d.slice/crio-2be9d01ccaf498e5d817e20f03a3ba840f69295797f79d0ce74f54c31b2f025e WatchSource:0}: Error finding container 2be9d01ccaf498e5d817e20f03a3ba840f69295797f79d0ce74f54c31b2f025e: Status 404 returned error can't find the container with id 2be9d01ccaf498e5d817e20f03a3ba840f69295797f79d0ce74f54c31b2f025e
Jul 07 00:29:28.138254 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:28.138208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" event={"ID":"a861cc6a-4cb7-4e66-ada8-051c937c6d0d","Type":"ContainerStarted","Data":"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"}
Jul 07 00:29:28.138431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:28.138267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" event={"ID":"a861cc6a-4cb7-4e66-ada8-051c937c6d0d","Type":"ContainerStarted","Data":"2be9d01ccaf498e5d817e20f03a3ba840f69295797f79d0ce74f54c31b2f025e"}
Jul 07 00:29:28.138431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:28.138290    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:28.156052 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:28.155984    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" podStartSLOduration=1.805847687 podStartE2EDuration="2.155969063s" totalImagesPullingTime="350.121376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:28.154709594 +0000 UTC m=+1970.558105667" watchObservedRunningTime="2026-07-07 00:29:28.155969063 +0000 UTC m=+1970.559365136"
Jul 07 00:29:34.149439 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:34.149364    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:39.432153 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.432120    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:39.432611 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.432361    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" podUID="fcb48510-adc9-4f07-bd0c-afdbc6371685" containerName="maas-api" containerID="cri-o://ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10" gracePeriod=30
Jul 07 00:29:39.683906 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.683834    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:39.858769 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.858737    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") pod \"fcb48510-adc9-4f07-bd0c-afdbc6371685\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") "
Jul 07 00:29:39.859064 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.858829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w45p\" (UniqueName: \"kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p\") pod \"fcb48510-adc9-4f07-bd0c-afdbc6371685\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") "
Jul 07 00:29:39.859064 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.858878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca\") pod \"fcb48510-adc9-4f07-bd0c-afdbc6371685\" (UID: \"fcb48510-adc9-4f07-bd0c-afdbc6371685\") "
Jul 07 00:29:39.859303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.859275    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcb48510-adc9-4f07-bd0c-afdbc6371685" (UID: "fcb48510-adc9-4f07-bd0c-afdbc6371685"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:29:39.861070 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.861038    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p" (OuterVolumeSpecName: "kube-api-access-5w45p") pod "fcb48510-adc9-4f07-bd0c-afdbc6371685" (UID: "fcb48510-adc9-4f07-bd0c-afdbc6371685"). InnerVolumeSpecName "kube-api-access-5w45p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:39.861163 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.861035    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcb48510-adc9-4f07-bd0c-afdbc6371685" (UID: "fcb48510-adc9-4f07-bd0c-afdbc6371685"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:29:39.960135 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.960049    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w45p\" (UniqueName: \"kubernetes.io/projected/fcb48510-adc9-4f07-bd0c-afdbc6371685-kube-api-access-5w45p\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:39.960135 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.960086    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcb48510-adc9-4f07-bd0c-afdbc6371685-openshift-service-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:39.960135 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:39.960101    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcb48510-adc9-4f07-bd0c-afdbc6371685-maas-api-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:40.198544 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.198503    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcb48510-adc9-4f07-bd0c-afdbc6371685" containerID="ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10" exitCode=0
Jul 07 00:29:40.198710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.198555    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"
Jul 07 00:29:40.198710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.198559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" event={"ID":"fcb48510-adc9-4f07-bd0c-afdbc6371685","Type":"ContainerDied","Data":"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"}
Jul 07 00:29:40.198710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.198619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc" event={"ID":"fcb48510-adc9-4f07-bd0c-afdbc6371685","Type":"ContainerDied","Data":"3939e39e631aa5d4ce2640f6347532ca81895733d9975eca827f27123d378834"}
Jul 07 00:29:40.198710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.198643    2581 scope.go:117] "RemoveContainer" containerID="ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"
Jul 07 00:29:40.207880 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.207844    2581 scope.go:117] "RemoveContainer" containerID="ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"
Jul 07 00:29:40.208119 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:40.208104    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10\": container with ID starting with ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10 not found: ID does not exist" containerID="ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"
Jul 07 00:29:40.208171 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.208126    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10"} err="failed to get container status \"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10\": rpc error: code = NotFound desc = could not find container \"ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10\": container with ID starting with ba30c81b5b547b112d2b3c492856b748f36fc9b5967d614162ef2bc6e2517a10 not found: ID does not exist"
Jul 07 00:29:40.220173 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.220150    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:40.224896 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.224873    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-2bebf4-8475776f98-98wnc"]
Jul 07 00:29:40.558839 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:40.558752    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcb48510-adc9-4f07-bd0c-afdbc6371685" path="/var/lib/kubelet/pods/fcb48510-adc9-4f07-bd0c-afdbc6371685/volumes"
Jul 07 00:29:47.395896 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.395837    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:47.396339 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.396081    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" podUID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" containerName="maas-api" containerID="cri-o://8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae" gracePeriod=30
Jul 07 00:29:47.645739 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.645714    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:47.731178 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.731146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") pod \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") "
Jul 07 00:29:47.731365 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.731225    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca\") pod \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") "
Jul 07 00:29:47.731365 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.731264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtmxl\" (UniqueName: \"kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl\") pod \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\" (UID: \"a861cc6a-4cb7-4e66-ada8-051c937c6d0d\") "
Jul 07 00:29:47.731650 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.731624    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a861cc6a-4cb7-4e66-ada8-051c937c6d0d" (UID: "a861cc6a-4cb7-4e66-ada8-051c937c6d0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:29:47.733376 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.733352    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl" (OuterVolumeSpecName: "kube-api-access-rtmxl") pod "a861cc6a-4cb7-4e66-ada8-051c937c6d0d" (UID: "a861cc6a-4cb7-4e66-ada8-051c937c6d0d"). InnerVolumeSpecName "kube-api-access-rtmxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:47.733497 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.733473    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a861cc6a-4cb7-4e66-ada8-051c937c6d0d" (UID: "a861cc6a-4cb7-4e66-ada8-051c937c6d0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:29:47.832978 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.832940    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-openshift-service-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:47.832978 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.832971    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtmxl\" (UniqueName: \"kubernetes.io/projected/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-kube-api-access-rtmxl\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:47.832978 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:47.832981    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a861cc6a-4cb7-4e66-ada8-051c937c6d0d-maas-api-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:29:48.243116 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.243084    2581 generic.go:356] "Generic (PLEG): container finished" podID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" containerID="8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae" exitCode=0
Jul 07 00:29:48.243283 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.243127    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"
Jul 07 00:29:48.243283 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.243160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" event={"ID":"a861cc6a-4cb7-4e66-ada8-051c937c6d0d","Type":"ContainerDied","Data":"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"}
Jul 07 00:29:48.243283 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.243202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48" event={"ID":"a861cc6a-4cb7-4e66-ada8-051c937c6d0d","Type":"ContainerDied","Data":"2be9d01ccaf498e5d817e20f03a3ba840f69295797f79d0ce74f54c31b2f025e"}
Jul 07 00:29:48.243283 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.243223    2581 scope.go:117] "RemoveContainer" containerID="8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"
Jul 07 00:29:48.252041 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.252022    2581 scope.go:117] "RemoveContainer" containerID="8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"
Jul 07 00:29:48.252291 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:48.252275    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae\": container with ID starting with 8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae not found: ID does not exist" containerID="8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"
Jul 07 00:29:48.252331 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.252301    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae"} err="failed to get container status \"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae\": rpc error: code = NotFound desc = could not find container \"8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae\": container with ID starting with 8bac43270189d9443abdd916471a80656fdd8b39f00e733d702e74195d70f7ae not found: ID does not exist"
Jul 07 00:29:48.264739 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.264717    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:48.268433 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.268410    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7e3541-66b878c78f-4fk48"]
Jul 07 00:29:48.558183 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:48.558103    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" path="/var/lib/kubelet/pods/a861cc6a-4cb7-4e66-ada8-051c937c6d0d/volumes"
Jul 07 00:29:51.617394 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:51.617355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:51.706186 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:51.706142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:53.310199 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:53.310137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:53.383510 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:53.383475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:53.616008 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:53.615925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:53.877093 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:53.876998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:54.288774 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:54.288740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:54.565763 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:54.565655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:54.963750 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:54.963651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.004944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.004901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.078256 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078220    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:29:58.078692 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078674    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb48510-adc9-4f07-bd0c-afdbc6371685" containerName="maas-api"
Jul 07 00:29:58.078801 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078695    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb48510-adc9-4f07-bd0c-afdbc6371685" containerName="maas-api"
Jul 07 00:29:58.078801 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078707    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" containerName="maas-api"
Jul 07 00:29:58.078801 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078715    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" containerName="maas-api"
Jul 07 00:29:58.078977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078809    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a861cc6a-4cb7-4e66-ada8-051c937c6d0d" containerName="maas-api"
Jul 07 00:29:58.078977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.078828    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb48510-adc9-4f07-bd0c-afdbc6371685" containerName="maas-api"
Jul 07 00:29:58.083375 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.083352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.085827 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.085798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-ea3965-serving-cert\""
Jul 07 00:29:58.085979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.085960    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g89gr\""
Jul 07 00:29:58.091104 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.091081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:29:58.098221 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.098194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.136164 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.136137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.136305 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.136226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xhfm\" (UniqueName: \"kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.136305 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.136260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.168418 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.168385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.236966 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.236936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xhfm\" (UniqueName: \"kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.237138 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.236980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.237138 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.237018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.237138 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:58.237111    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-ea3965-serving-cert: secret "maas-api-e2e-shared-a-ea3965-serving-cert" not found
Jul 07 00:29:58.237305 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:58.237163    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls podName:0e9d2101-ff79-41ed-b75d-6358b7c48bce nodeName:}" failed. No retries permitted until 2026-07-07 00:29:58.73714865 +0000 UTC m=+2001.140544701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls") pod "maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" (UID: "0e9d2101-ff79-41ed-b75d-6358b7c48bce") : secret "maas-api-e2e-shared-a-ea3965-serving-cert" not found
Jul 07 00:29:58.237686 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.237663    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.249077 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.249055    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xhfm\" (UniqueName: \"kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.680079 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.680045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.740832 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.740796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:58.741033 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:58.740951    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-ea3965-serving-cert: secret "maas-api-e2e-shared-a-ea3965-serving-cert" not found
Jul 07 00:29:58.741033 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:29:58.741011    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls podName:0e9d2101-ff79-41ed-b75d-6358b7c48bce nodeName:}" failed. No retries permitted until 2026-07-07 00:29:59.740995811 +0000 UTC m=+2002.144391862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls") pod "maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" (UID: "0e9d2101-ff79-41ed-b75d-6358b7c48bce") : secret "maas-api-e2e-shared-a-ea3965-serving-cert" not found
Jul 07 00:29:58.780046 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.780009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:58.850901 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:58.850851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:59.751042 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:59.751008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:59.753370 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:59.753336    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") " pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:29:59.896499 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:29:59.896460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:30:00.033783 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.033757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:30:00.035484 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:30:00.035457    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9d2101_ff79_41ed_b75d_6358b7c48bce.slice/crio-44bf53190aca625c7e91a1278cc33d5ab8821520883d1bc7394b3390cd9ae7c3 WatchSource:0}: Error finding container 44bf53190aca625c7e91a1278cc33d5ab8821520883d1bc7394b3390cd9ae7c3: Status 404 returned error can't find the container with id 44bf53190aca625c7e91a1278cc33d5ab8821520883d1bc7394b3390cd9ae7c3
Jul 07 00:30:00.149605 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.149573    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-jpgdq"]
Jul 07 00:30:00.154358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.154337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:00.169540 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.169491    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-jpgdq"]
Jul 07 00:30:00.255282 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.255247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltwg\" (UniqueName: \"kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg\") pod \"maas-api-key-cleanup-29723070-jpgdq\" (UID: \"2f890ca2-7318-4e4c-be19-4914dbae9fde\") " pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:00.303412 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.303336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" event={"ID":"0e9d2101-ff79-41ed-b75d-6358b7c48bce","Type":"ContainerStarted","Data":"44bf53190aca625c7e91a1278cc33d5ab8821520883d1bc7394b3390cd9ae7c3"}
Jul 07 00:30:00.356580 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.356548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltwg\" (UniqueName: \"kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg\") pod \"maas-api-key-cleanup-29723070-jpgdq\" (UID: \"2f890ca2-7318-4e4c-be19-4914dbae9fde\") " pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:00.365660 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.365637    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltwg\" (UniqueName: \"kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg\") pod \"maas-api-key-cleanup-29723070-jpgdq\" (UID: \"2f890ca2-7318-4e4c-be19-4914dbae9fde\") " pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:00.464796 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.464759    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:00.599336 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:00.599283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723070-jpgdq"]
Jul 07 00:30:00.601247 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:30:00.601219    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f890ca2_7318_4e4c_be19_4914dbae9fde.slice/crio-9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4 WatchSource:0}: Error finding container 9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4: Status 404 returned error can't find the container with id 9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4
Jul 07 00:30:01.309810 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.309771    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" event={"ID":"0e9d2101-ff79-41ed-b75d-6358b7c48bce","Type":"ContainerStarted","Data":"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"}
Jul 07 00:30:01.310257 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.309867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:30:01.311258 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.311236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq" event={"ID":"2f890ca2-7318-4e4c-be19-4914dbae9fde","Type":"ContainerStarted","Data":"8cae41465f38f256ade55553377dd205932450fcd4efce5c5e128190e7b6f3b1"}
Jul 07 00:30:01.311367 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.311277    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq" event={"ID":"2f890ca2-7318-4e4c-be19-4914dbae9fde","Type":"ContainerStarted","Data":"9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4"}
Jul 07 00:30:01.331190 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.331146    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" podStartSLOduration=3.003249851 podStartE2EDuration="3.331132806s" totalImagesPullingTime="327.882955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:01.329714127 +0000 UTC m=+2003.733110200" watchObservedRunningTime="2026-07-07 00:30:01.331132806 +0000 UTC m=+2003.734528878"
Jul 07 00:30:01.348234 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:01.348193    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq" podStartSLOduration=1.348179292 podStartE2EDuration="1.348179292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:01.346828276 +0000 UTC m=+2003.750224380" watchObservedRunningTime="2026-07-07 00:30:01.348179292 +0000 UTC m=+2003.751575364"
Jul 07 00:30:02.317998 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:02.317968    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f890ca2-7318-4e4c-be19-4914dbae9fde" containerID="8cae41465f38f256ade55553377dd205932450fcd4efce5c5e128190e7b6f3b1" exitCode=0
Jul 07 00:30:02.318399 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:02.318054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq" event={"ID":"2f890ca2-7318-4e4c-be19-4914dbae9fde","Type":"ContainerDied","Data":"8cae41465f38f256ade55553377dd205932450fcd4efce5c5e128190e7b6f3b1"}
Jul 07 00:30:02.542102 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:02.542066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:03.001070 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.001032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:03.219445 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.219411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:03.392187 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.392144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:03.477984 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.477962    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:03.590615 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.590536    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltwg\" (UniqueName: \"kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg\") pod \"2f890ca2-7318-4e4c-be19-4914dbae9fde\" (UID: \"2f890ca2-7318-4e4c-be19-4914dbae9fde\") "
Jul 07 00:30:03.592596 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.592565    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg" (OuterVolumeSpecName: "kube-api-access-wltwg") pod "2f890ca2-7318-4e4c-be19-4914dbae9fde" (UID: "2f890ca2-7318-4e4c-be19-4914dbae9fde"). InnerVolumeSpecName "kube-api-access-wltwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:30:03.691678 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:03.691621    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltwg\" (UniqueName: \"kubernetes.io/projected/2f890ca2-7318-4e4c-be19-4914dbae9fde-kube-api-access-wltwg\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:30:04.329628 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:04.329593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq" event={"ID":"2f890ca2-7318-4e4c-be19-4914dbae9fde","Type":"ContainerDied","Data":"9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4"}
Jul 07 00:30:04.329804 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:04.329641    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9da47210b3f43ed6f1c3db973d8507e564ed8d455cfba8a1b89e9d099f6727e4"
Jul 07 00:30:04.329804 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:04.329610    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723070-jpgdq"
Jul 07 00:30:07.218208 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.218162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:07.307620 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.307581    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:30:07.308079 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.308062    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f890ca2-7318-4e4c-be19-4914dbae9fde" containerName="cleanup"
Jul 07 00:30:07.308186 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.308082    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f890ca2-7318-4e4c-be19-4914dbae9fde" containerName="cleanup"
Jul 07 00:30:07.308186 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.308163    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f890ca2-7318-4e4c-be19-4914dbae9fde" containerName="cleanup"
Jul 07 00:30:07.310662 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.310634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:07.311409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.311393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.313958 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.313936    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-e15ad3-serving-cert\""
Jul 07 00:30:07.323461 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.323442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:30:07.332669 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.332645    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:30:07.387452 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.387404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:07.428294 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.428265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.428431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.428356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvj6c\" (UniqueName: \"kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.428431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.428413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.529422 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.529327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.529422 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.529407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.529604 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.529457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvj6c\" (UniqueName: \"kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.529604 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:30:07.529553    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-e15ad3-serving-cert: secret "maas-api-e2e-shared-b-e15ad3-serving-cert" not found
Jul 07 00:30:07.529674 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:30:07.529625    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls podName:f9757c2f-58a3-46ce-b894-cac099990602 nodeName:}" failed. No retries permitted until 2026-07-07 00:30:08.029607609 +0000 UTC m=+2010.433003675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls") pod "maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" (UID: "f9757c2f-58a3-46ce-b894-cac099990602") : secret "maas-api-e2e-shared-b-e15ad3-serving-cert" not found
Jul 07 00:30:07.529965 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.529948    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.548075 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.548049    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvj6c\" (UniqueName: \"kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:07.638406 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.638372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:07.913926 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:07.913814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:08.016832 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:08.016791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:08.035380 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:08.035356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:08.037742 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:08.037716    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") pod \"maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") " pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:08.222676 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:08.222636    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:08.355521 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:08.355496    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:30:08.357591 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:30:08.357553    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9757c2f_58a3_46ce_b894_cac099990602.slice/crio-83bfe1e0d3c11ba0dba5e1fd82966755c8a7c73912228d245f20c4405bb1ae44 WatchSource:0}: Error finding container 83bfe1e0d3c11ba0dba5e1fd82966755c8a7c73912228d245f20c4405bb1ae44: Status 404 returned error can't find the container with id 83bfe1e0d3c11ba0dba5e1fd82966755c8a7c73912228d245f20c4405bb1ae44
Jul 07 00:30:09.358393 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:09.358359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" event={"ID":"f9757c2f-58a3-46ce-b894-cac099990602","Type":"ContainerStarted","Data":"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"}
Jul 07 00:30:09.358864 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:09.358412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" event={"ID":"f9757c2f-58a3-46ce-b894-cac099990602","Type":"ContainerStarted","Data":"83bfe1e0d3c11ba0dba5e1fd82966755c8a7c73912228d245f20c4405bb1ae44"}
Jul 07 00:30:09.358864 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:09.358440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:09.377161 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:09.377109    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" podStartSLOduration=1.9367584770000001 podStartE2EDuration="2.377093319s" totalImagesPullingTime="440.334842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:09.375114936 +0000 UTC m=+2011.778511010" watchObservedRunningTime="2026-07-07 00:30:09.377093319 +0000 UTC m=+2011.780489392"
Jul 07 00:30:11.747668 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:11.747636    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:15.376943 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:15.376909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:30:19.409099 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:19.409067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:19.510088 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:19.510039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:23.790782 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.790744    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:30:23.794985 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.794964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.797731 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.797712    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"default-dockercfg-8w5rx\""
Jul 07 00:30:23.798798 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.798783    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"openshift-service-ca.crt\""
Jul 07 00:30:23.798886 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.798798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"auth-test-model-ea3965-kserve-self-signed-certs\""
Jul 07 00:30:23.798886 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.798782    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"kube-root-ca.crt\""
Jul 07 00:30:23.802708 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.802686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:30:23.813113 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.813087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:23.889093 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.889247 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.889247 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.889358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.889358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqxmk\" (UniqueName: \"kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.889358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.889322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990070 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990240 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqxmk\" (UniqueName: \"kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990240 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990240 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990240 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990373 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990608 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990587    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990647 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990608    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.990733 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.990716    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.992537 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.992515    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.992673 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.992655    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:23.999330 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:23.999305    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqxmk\" (UniqueName: \"kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk\") pod \"auth-test-model-ea3965-kserve-76ffb4787f-zbz9c\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") " pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:24.105635 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:24.105542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:24.243778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:24.243754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:30:24.245753 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:30:24.245725    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod818fb607_4f9d_4c28_ba48_b1a25a515733.slice/crio-08ddd60a276ca4f1e8046b4818ac4adcae0be581cf83bf0ab20e9cad32dd363f WatchSource:0}: Error finding container 08ddd60a276ca4f1e8046b4818ac4adcae0be581cf83bf0ab20e9cad32dd363f: Status 404 returned error can't find the container with id 08ddd60a276ca4f1e8046b4818ac4adcae0be581cf83bf0ab20e9cad32dd363f
Jul 07 00:30:24.448786 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:24.448687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" event={"ID":"818fb607-4f9d-4c28-ba48-b1a25a515733","Type":"ContainerStarted","Data":"c521072eb139ac4a4254f7b827e35564b8d9acb06e2a544cb88f5b51dd44943f"}
Jul 07 00:30:24.448786 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:24.448738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" event={"ID":"818fb607-4f9d-4c28-ba48-b1a25a515733","Type":"ContainerStarted","Data":"08ddd60a276ca4f1e8046b4818ac4adcae0be581cf83bf0ab20e9cad32dd363f"}
Jul 07 00:30:24.470181 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:24.470127    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" podStartSLOduration=1.470108718 podStartE2EDuration="1.470108718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:24.468703139 +0000 UTC m=+2026.872099213" watchObservedRunningTime="2026-07-07 00:30:24.470108718 +0000 UTC m=+2026.873504824"
Jul 07 00:30:28.779579 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:28.779543    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:28.882486 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:28.882452    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:34.105979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:34.105945    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:34.105979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:34.105995    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:34.118362 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:34.118333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:34.511383 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:34.511353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:30:53.094569 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.094535    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:30:53.098787 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.098184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.101055 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.101027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"openshift-service-ca.crt\""
Jul 07 00:30:53.102237 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.102217    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"auth-test-model-e15ad3-kserve-self-signed-certs\""
Jul 07 00:30:53.102339 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.102223    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"kube-root-ca.crt\""
Jul 07 00:30:53.102404 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.102375    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"default-dockercfg-8bxf8\""
Jul 07 00:30:53.107489 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.107468    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:30:53.121235 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.121199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:53.168549 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.168549 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.168778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnsfg\" (UniqueName: \"kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.168778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.168778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.168778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.168773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269537 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.269732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.269689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnsfg\" (UniqueName: \"kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.270056 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.270030    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.270119 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.270043    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.270170 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.270109    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.272025 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.272004    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.272319 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.272299    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.278383 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.278361    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnsfg\" (UniqueName: \"kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg\") pod \"auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") " pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.408341 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.408254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:30:53.545403 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.545378    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:30:53.547629 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:30:53.547604    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56798055_2e7b_453f_bb7b_abbdc1c81962.slice/crio-9c7e1e9ae7fc0d3e3e2dee87b5107bc1b0e17bd6434a5d3e1d798757851ced67 WatchSource:0}: Error finding container 9c7e1e9ae7fc0d3e3e2dee87b5107bc1b0e17bd6434a5d3e1d798757851ced67: Status 404 returned error can't find the container with id 9c7e1e9ae7fc0d3e3e2dee87b5107bc1b0e17bd6434a5d3e1d798757851ced67
Jul 07 00:30:53.596098 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:53.596069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" event={"ID":"56798055-2e7b-453f-bb7b-abbdc1c81962","Type":"ContainerStarted","Data":"9c7e1e9ae7fc0d3e3e2dee87b5107bc1b0e17bd6434a5d3e1d798757851ced67"}
Jul 07 00:30:54.602759 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:54.602724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" event={"ID":"56798055-2e7b-453f-bb7b-abbdc1c81962","Type":"ContainerStarted","Data":"0a83b18043b5f72b06411fb2c3c404b9d38a73f58fcf1578362209800f82b895"}
Jul 07 00:30:54.619821 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:54.619768    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" podStartSLOduration=1.6197529510000002 podStartE2EDuration="1.619752951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:54.618683276 +0000 UTC m=+2057.022079349" watchObservedRunningTime="2026-07-07 00:30:54.619752951 +0000 UTC m=+2057.023149023"
Jul 07 00:30:59.067869 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:30:59.067813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:03.408792 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:31:03.408747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:31:03.408792 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:31:03.408798    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:31:03.421320 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:31:03.421294    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:31:03.659607 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:31:03.659522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:32:33.050352 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:32:33.050246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:04.186405 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:04.186373    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:33:04.257426 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:04.257392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:05.079538 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:05.079500    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:33:05.106023 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:05.105984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:19.206030 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.205987    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" podUID="818fb607-4f9d-4c28-ba48-b1a25a515733" containerName="main" containerID="cri-o://c521072eb139ac4a4254f7b827e35564b8d9acb06e2a544cb88f5b51dd44943f" gracePeriod=45
Jul 07 00:33:19.329977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.329950    2581 generic.go:356] "Generic (PLEG): container finished" podID="818fb607-4f9d-4c28-ba48-b1a25a515733" containerID="c521072eb139ac4a4254f7b827e35564b8d9acb06e2a544cb88f5b51dd44943f" exitCode=0
Jul 07 00:33:19.330103 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.329986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" event={"ID":"818fb607-4f9d-4c28-ba48-b1a25a515733","Type":"ContainerDied","Data":"c521072eb139ac4a4254f7b827e35564b8d9acb06e2a544cb88f5b51dd44943f"}
Jul 07 00:33:19.469449 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.469419    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:33:19.593684 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.593684 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.593944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.593944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593820    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqxmk\" (UniqueName: \"kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.593944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.593944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593930    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm\") pod \"818fb607-4f9d-4c28-ba48-b1a25a515733\" (UID: \"818fb607-4f9d-4c28-ba48-b1a25a515733\") "
Jul 07 00:33:19.594140 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.593969    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:19.594140 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.594048    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache" (OuterVolumeSpecName: "model-cache") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:19.594140 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.594101    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home" (OuterVolumeSpecName: "home") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:19.594315 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.594284    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:19.594315 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.594303    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:19.594315 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.594313    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:19.596119 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.596094    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk" (OuterVolumeSpecName: "kube-api-access-kqxmk") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "kube-api-access-kqxmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:33:19.596213 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.596096    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm" (OuterVolumeSpecName: "dshm") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:19.596213 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.596086    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "818fb607-4f9d-4c28-ba48-b1a25a515733" (UID: "818fb607-4f9d-4c28-ba48-b1a25a515733"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:33:19.695453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.695414    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqxmk\" (UniqueName: \"kubernetes.io/projected/818fb607-4f9d-4c28-ba48-b1a25a515733-kube-api-access-kqxmk\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:19.695453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.695450    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818fb607-4f9d-4c28-ba48-b1a25a515733-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:19.695453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:19.695461    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/818fb607-4f9d-4c28-ba48-b1a25a515733-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.100897 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.100836    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" podUID="56798055-2e7b-453f-bb7b-abbdc1c81962" containerName="main" containerID="cri-o://0a83b18043b5f72b06411fb2c3c404b9d38a73f58fcf1578362209800f82b895" gracePeriod=45
Jul 07 00:33:20.332498 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.332458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:20.343954 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.343926    2581 generic.go:356] "Generic (PLEG): container finished" podID="56798055-2e7b-453f-bb7b-abbdc1c81962" containerID="0a83b18043b5f72b06411fb2c3c404b9d38a73f58fcf1578362209800f82b895" exitCode=0
Jul 07 00:33:20.344093 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.343995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" event={"ID":"56798055-2e7b-453f-bb7b-abbdc1c81962","Type":"ContainerDied","Data":"0a83b18043b5f72b06411fb2c3c404b9d38a73f58fcf1578362209800f82b895"}
Jul 07 00:33:20.345522 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.345497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c" event={"ID":"818fb607-4f9d-4c28-ba48-b1a25a515733","Type":"ContainerDied","Data":"08ddd60a276ca4f1e8046b4818ac4adcae0be581cf83bf0ab20e9cad32dd363f"}
Jul 07 00:33:20.345636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.345534    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"
Jul 07 00:33:20.345636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.345544    2581 scope.go:117] "RemoveContainer" containerID="c521072eb139ac4a4254f7b827e35564b8d9acb06e2a544cb88f5b51dd44943f"
Jul 07 00:33:20.361505 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.361484    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:33:20.371912 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.371883    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:33:20.375804 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.375781    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/auth-test-model-ea3965-kserve-76ffb4787f-zbz9c"]
Jul 07 00:33:20.504015 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.503981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504196 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504052    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504196 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504087    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504196 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnsfg\" (UniqueName: \"kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504196 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504251    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home\") pod \"56798055-2e7b-453f-bb7b-abbdc1c81962\" (UID: \"56798055-2e7b-453f-bb7b-abbdc1c81962\") "
Jul 07 00:33:20.504431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504284    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache" (OuterVolumeSpecName: "model-cache") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:20.504431 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504355    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:20.504594 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504571    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home" (OuterVolumeSpecName: "home") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:20.504725 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504712    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.504782 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504733    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.504782 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.504773    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.506490 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.506468    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg" (OuterVolumeSpecName: "kube-api-access-rnsfg") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "kube-api-access-rnsfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:33:20.506490 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.506482    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm" (OuterVolumeSpecName: "dshm") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:33:20.506662 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.506584    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56798055-2e7b-453f-bb7b-abbdc1c81962" (UID: "56798055-2e7b-453f-bb7b-abbdc1c81962"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:33:20.558455 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.558426    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818fb607-4f9d-4c28-ba48-b1a25a515733" path="/var/lib/kubelet/pods/818fb607-4f9d-4c28-ba48-b1a25a515733/volumes"
Jul 07 00:33:20.605916 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.605878    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56798055-2e7b-453f-bb7b-abbdc1c81962-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.605916 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.605906    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnsfg\" (UniqueName: \"kubernetes.io/projected/56798055-2e7b-453f-bb7b-abbdc1c81962-kube-api-access-rnsfg\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:20.605916 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:20.605916    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56798055-2e7b-453f-bb7b-abbdc1c81962-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:33:21.351645 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:21.351618    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"
Jul 07 00:33:21.352145 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:21.351661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc" event={"ID":"56798055-2e7b-453f-bb7b-abbdc1c81962","Type":"ContainerDied","Data":"9c7e1e9ae7fc0d3e3e2dee87b5107bc1b0e17bd6434a5d3e1d798757851ced67"}
Jul 07 00:33:21.352145 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:21.351712    2581 scope.go:117] "RemoveContainer" containerID="0a83b18043b5f72b06411fb2c3c404b9d38a73f58fcf1578362209800f82b895"
Jul 07 00:33:21.367930 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:21.367906    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:33:21.371292 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:21.371270    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/auth-test-model-e15ad3-kserve-5488f49ff5-fxgbc"]
Jul 07 00:33:22.557787 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:22.557752    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56798055-2e7b-453f-bb7b-abbdc1c81962" path="/var/lib/kubelet/pods/56798055-2e7b-453f-bb7b-abbdc1c81962/volumes"
Jul 07 00:33:34.883088 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883490    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818fb607-4f9d-4c28-ba48-b1a25a515733" containerName="main"
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883503    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="818fb607-4f9d-4c28-ba48-b1a25a515733" containerName="main"
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883515    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56798055-2e7b-453f-bb7b-abbdc1c81962" containerName="main"
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883521    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56798055-2e7b-453f-bb7b-abbdc1c81962" containerName="main"
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883584    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56798055-2e7b-453f-bb7b-abbdc1c81962" containerName="main"
Jul 07 00:33:34.883700 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.883596    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="818fb607-4f9d-4c28-ba48-b1a25a515733" containerName="main"
Jul 07 00:33:34.888220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.888197    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:34.892041 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.891997    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"kube-root-ca.crt\""
Jul 07 00:33:34.892041 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.892027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"sub-test-model-ea3965-kserve-self-signed-certs\""
Jul 07 00:33:34.892273 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.892120    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"openshift-service-ca.crt\""
Jul 07 00:33:34.892489 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.892469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"default-dockercfg-8w5rx\""
Jul 07 00:33:34.893620 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.893586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:33:34.915832 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:34.915787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:35.049100 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.049276 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.049276 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.049276 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.049276 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.049425 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.049281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xtpl\" (UniqueName: \"kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.149940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.149982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xtpl\" (UniqueName: \"kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150360 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150360 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150474 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150474 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150427    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.150568 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.150489    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.152306 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.152275    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.152567 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.152551    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.158269 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.158245    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xtpl\" (UniqueName: \"kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl\") pod \"sub-test-model-ea3965-kserve-7f86df5f57-wvl6l\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") " pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.200210 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.200173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:35.333388 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.333348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:33:35.336475 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:33:35.336448    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod991e72fc_0b02_41bd_ae2d_7ca0ecd1bfc2.slice/crio-00542640098e8f15e44e24213e00569e35efb1494fb8cea3c7ea4542a259b738 WatchSource:0}: Error finding container 00542640098e8f15e44e24213e00569e35efb1494fb8cea3c7ea4542a259b738: Status 404 returned error can't find the container with id 00542640098e8f15e44e24213e00569e35efb1494fb8cea3c7ea4542a259b738
Jul 07 00:33:35.429598 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.429509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" event={"ID":"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2","Type":"ContainerStarted","Data":"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"}
Jul 07 00:33:35.429598 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.429561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" event={"ID":"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2","Type":"ContainerStarted","Data":"00542640098e8f15e44e24213e00569e35efb1494fb8cea3c7ea4542a259b738"}
Jul 07 00:33:35.447445 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:35.447393    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" podStartSLOduration=1.4473773890000001 podStartE2EDuration="1.447377389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:33:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:33:35.444915986 +0000 UTC m=+2217.848312060" watchObservedRunningTime="2026-07-07 00:33:35.447377389 +0000 UTC m=+2217.850773462"
Jul 07 00:33:45.200537 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:45.200488    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:45.200968 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:45.200568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:45.213446 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:45.213416    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:33:45.490471 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:33:45.490447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:34:00.724127 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:00.724046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:38.668233 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:38.668192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:38.972434 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:38.972399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:51.594363 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:51.594329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:51.701226 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:51.701198    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:57.215989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:57.215948    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:34:57.234112 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:57.234078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:34:58.123559 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:34:58.123531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:01.596597 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.596564    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:35:01.600453 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.600435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.603325 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.603301    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ea3965\"/\"rate-test-model-ea3965-kserve-self-signed-certs\""
Jul 07 00:35:01.610289 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.610260    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:35:01.627551 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.627523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:01.695989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.695952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.695989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.695992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.696205 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.696055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.696205 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.696076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.696205 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.696188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.696302 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.696233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlrbs\" (UniqueName: \"kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797034 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.796998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797034 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797244 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797244 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797244 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797411 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlrbs\" (UniqueName: \"kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797467 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797436    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797492    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.797524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.797514    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.799465 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.799446    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.799708 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.799690    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.805641 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.805621    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlrbs\" (UniqueName: \"kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs\") pod \"rate-test-model-ea3965-kserve-9559ff7b7-hwq76\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") " pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:01.910794 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:01.910698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:02.044793 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:02.044769    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:35:02.046309 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:35:02.046281    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod111fabd4_6015_41a0_b0b4_577e3fbd5e9f.slice/crio-a6988d1f9431ea1a1e4671afdc3c2f08276c627d518e02f9f43d1ffe461c3c43 WatchSource:0}: Error finding container a6988d1f9431ea1a1e4671afdc3c2f08276c627d518e02f9f43d1ffe461c3c43: Status 404 returned error can't find the container with id a6988d1f9431ea1a1e4671afdc3c2f08276c627d518e02f9f43d1ffe461c3c43
Jul 07 00:35:02.862194 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:02.862158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" event={"ID":"111fabd4-6015-41a0-b0b4-577e3fbd5e9f","Type":"ContainerStarted","Data":"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"}
Jul 07 00:35:02.862660 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:02.862202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" event={"ID":"111fabd4-6015-41a0-b0b4-577e3fbd5e9f","Type":"ContainerStarted","Data":"a6988d1f9431ea1a1e4671afdc3c2f08276c627d518e02f9f43d1ffe461c3c43"}
Jul 07 00:35:02.880734 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:02.880684    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" podStartSLOduration=1.8806697369999998 podStartE2EDuration="1.880669737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:35:02.87853207 +0000 UTC m=+2305.281928177" watchObservedRunningTime="2026-07-07 00:35:02.880669737 +0000 UTC m=+2305.284065810"
Jul 07 00:35:11.911669 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:11.911624    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:11.912185 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:11.911686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:11.924397 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:11.924368    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:12.236570 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.236526    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" podUID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" containerName="main" containerID="cri-o://9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18" gracePeriod=45
Jul 07 00:35:12.494735 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.494675    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:35:12.605397 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605397 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605427    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605497    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xtpl\" (UniqueName: \"kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605648 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs\") pod \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\" (UID: \"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2\") "
Jul 07 00:35:12.605851 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605684    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache" (OuterVolumeSpecName: "model-cache") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:12.605851 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605695    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home" (OuterVolumeSpecName: "home") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:12.605851 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605816    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:12.606039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605961    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.606039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605982    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.606039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.605998    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.607664 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.607640    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:35:12.607664 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.607654    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm" (OuterVolumeSpecName: "dshm") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:12.607812 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.607711    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl" (OuterVolumeSpecName: "kube-api-access-5xtpl") pod "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" (UID: "991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2"). InnerVolumeSpecName "kube-api-access-5xtpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:35:12.706730 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.706691    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xtpl\" (UniqueName: \"kubernetes.io/projected/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-kube-api-access-5xtpl\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.706730 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.706724    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.706730 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.706735    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:35:12.911825 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.911712    2581 generic.go:356] "Generic (PLEG): container finished" podID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" containerID="9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18" exitCode=0
Jul 07 00:35:12.911825 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.911778    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"
Jul 07 00:35:12.911825 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.911794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" event={"ID":"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2","Type":"ContainerDied","Data":"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"}
Jul 07 00:35:12.912423 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.911842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l" event={"ID":"991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2","Type":"ContainerDied","Data":"00542640098e8f15e44e24213e00569e35efb1494fb8cea3c7ea4542a259b738"}
Jul 07 00:35:12.912423 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.911885    2581 scope.go:117] "RemoveContainer" containerID="9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"
Jul 07 00:35:12.921919 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.921895    2581 scope.go:117] "RemoveContainer" containerID="9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"
Jul 07 00:35:12.922206 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:35:12.922187    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18\": container with ID starting with 9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18 not found: ID does not exist" containerID="9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"
Jul 07 00:35:12.922277 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.922216    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18"} err="failed to get container status \"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18\": rpc error: code = NotFound desc = could not find container \"9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18\": container with ID starting with 9dda6cedf781fd1ffda251d467271fadd3bb12daed790633440598739f125a18 not found: ID does not exist"
Jul 07 00:35:12.924283 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.924263    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:35:12.932670 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.932646    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:35:12.936301 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:12.936277    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/sub-test-model-ea3965-kserve-7f86df5f57-wvl6l"]
Jul 07 00:35:14.559376 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:14.559340    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" path="/var/lib/kubelet/pods/991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2/volumes"
Jul 07 00:35:20.206957 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:20.206927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:20.303170 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:20.303135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:22.390641 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.390607    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:35:22.391083 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.391069    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" containerName="main"
Jul 07 00:35:22.391140 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.391085    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" containerName="main"
Jul 07 00:35:22.391182 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.391159    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="991e72fc-0b02-41bd-ae2d-7ca0ecd1bfc2" containerName="main"
Jul 07 00:35:22.394268 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.394250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.399139 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.399113    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"rate-test-model-e15ad3-kserve-self-signed-certs\""
Jul 07 00:35:22.400299 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.400275    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"default-dockercfg-8bxf8\""
Jul 07 00:35:22.400299 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.400280    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"kube-root-ca.crt\""
Jul 07 00:35:22.400486 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.400285    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e15ad3\"/\"openshift-service-ca.crt\""
Jul 07 00:35:22.404008 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.403986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.404008 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.404016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.404166 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.404045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.404237 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.404167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.404306 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.404287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.404359 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.404341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmg2\" (UniqueName: \"kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.412303 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.412274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:35:22.425884 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.425841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:22.505632 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.505800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmg2\" (UniqueName: \"kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.505800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.505800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.505800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.505800 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.505743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.506096 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.506050    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.506096 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.506072    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.506173 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.506149    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.507949 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.507927    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.508293 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.508271    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.513792 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.513773    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmg2\" (UniqueName: \"kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2\") pod \"rate-test-model-e15ad3-kserve-9776996cf-b7m5c\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") " pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.703678 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.703581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:22.835948 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.835922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:35:22.838040 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:35:22.838007    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe497a7d_2574_4d56_81d6_6dcdb7169223.slice/crio-d96450734d9ad07ddbf708e516f0aeb6d02769fb4498ff640a6a601e9a811fe9 WatchSource:0}: Error finding container d96450734d9ad07ddbf708e516f0aeb6d02769fb4498ff640a6a601e9a811fe9: Status 404 returned error can't find the container with id d96450734d9ad07ddbf708e516f0aeb6d02769fb4498ff640a6a601e9a811fe9
Jul 07 00:35:22.966432 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.966390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" event={"ID":"fe497a7d-2574-4d56-81d6-6dcdb7169223","Type":"ContainerStarted","Data":"082f214e304f3756d1d0f88044e911e3198bf60d23b0c3b5e8a93b54d9ae0113"}
Jul 07 00:35:22.966432 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.966433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" event={"ID":"fe497a7d-2574-4d56-81d6-6dcdb7169223","Type":"ContainerStarted","Data":"d96450734d9ad07ddbf708e516f0aeb6d02769fb4498ff640a6a601e9a811fe9"}
Jul 07 00:35:22.983672 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:22.983620    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" podStartSLOduration=0.983603964 podStartE2EDuration="983.603964ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:35:22.982202616 +0000 UTC m=+2325.385598693" watchObservedRunningTime="2026-07-07 00:35:22.983603964 +0000 UTC m=+2325.387000037"
Jul 07 00:35:29.822719 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:29.822625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:29.921353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:29.921302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:32.703921 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:32.703887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:32.704308 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:32.703935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:32.716191 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:32.716158    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:35:33.028250 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:35:33.028224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:36:06.220378 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:06.220344    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:36:06.240245 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:06.240217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:07.127302 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:07.127271    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:36:07.140004 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:07.139970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:21.240953 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.240906    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" podUID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" containerName="main" containerID="cri-o://5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8" gracePeriod=45
Jul 07 00:36:21.492940 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.492876    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:36:21.606632 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.606808 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606655    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.606808 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.606808 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606715    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.606808 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.607020 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606814    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlrbs\" (UniqueName: \"kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs\") pod \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\" (UID: \"111fabd4-6015-41a0-b0b4-577e3fbd5e9f\") "
Jul 07 00:36:21.607020 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.606937    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:21.607020 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.607000    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home" (OuterVolumeSpecName: "home") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:21.607177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.607027    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:21.607177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.607148    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:21.607177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.607162    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:21.607177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.607169    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:21.608923 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.608892    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm" (OuterVolumeSpecName: "dshm") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:21.609043 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.608935    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:36:21.609103 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.609041    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs" (OuterVolumeSpecName: "kube-api-access-mlrbs") pod "111fabd4-6015-41a0-b0b4-577e3fbd5e9f" (UID: "111fabd4-6015-41a0-b0b4-577e3fbd5e9f"). InnerVolumeSpecName "kube-api-access-mlrbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:21.708109 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.708069    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:21.708109 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.708101    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:21.708109 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:21.708116    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlrbs\" (UniqueName: \"kubernetes.io/projected/111fabd4-6015-41a0-b0b4-577e3fbd5e9f-kube-api-access-mlrbs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.147330 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.147269    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" podUID="fe497a7d-2574-4d56-81d6-6dcdb7169223" containerName="main" containerID="cri-o://082f214e304f3756d1d0f88044e911e3198bf60d23b0c3b5e8a93b54d9ae0113" gracePeriod=45
Jul 07 00:36:22.282574 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.282530    2581 generic.go:356] "Generic (PLEG): container finished" podID="fe497a7d-2574-4d56-81d6-6dcdb7169223" containerID="082f214e304f3756d1d0f88044e911e3198bf60d23b0c3b5e8a93b54d9ae0113" exitCode=0
Jul 07 00:36:22.283024 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.282599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" event={"ID":"fe497a7d-2574-4d56-81d6-6dcdb7169223","Type":"ContainerDied","Data":"082f214e304f3756d1d0f88044e911e3198bf60d23b0c3b5e8a93b54d9ae0113"}
Jul 07 00:36:22.284213 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.284187    2581 generic.go:356] "Generic (PLEG): container finished" podID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" containerID="5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8" exitCode=0
Jul 07 00:36:22.284353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.284236    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"
Jul 07 00:36:22.284353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.284269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" event={"ID":"111fabd4-6015-41a0-b0b4-577e3fbd5e9f","Type":"ContainerDied","Data":"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"}
Jul 07 00:36:22.284353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.284305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76" event={"ID":"111fabd4-6015-41a0-b0b4-577e3fbd5e9f","Type":"ContainerDied","Data":"a6988d1f9431ea1a1e4671afdc3c2f08276c627d518e02f9f43d1ffe461c3c43"}
Jul 07 00:36:22.284353 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.284324    2581 scope.go:117] "RemoveContainer" containerID="5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"
Jul 07 00:36:22.295371 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.294990    2581 scope.go:117] "RemoveContainer" containerID="5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"
Jul 07 00:36:22.295371 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:22.295234    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8\": container with ID starting with 5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8 not found: ID does not exist" containerID="5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"
Jul 07 00:36:22.295371 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.295257    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8"} err="failed to get container status \"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8\": rpc error: code = NotFound desc = could not find container \"5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8\": container with ID starting with 5772bb94ebed275dc8d5c40836c2e73e8adf458c62340714c90ded3bcaf944d8 not found: ID does not exist"
Jul 07 00:36:22.307744 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.307714    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:36:22.314066 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.314041    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ea3965/rate-test-model-ea3965-kserve-9559ff7b7-hwq76"]
Jul 07 00:36:22.401652 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.401592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:36:22.514745 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514711    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.514942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514768    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.514942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.514942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514841    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.514942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pmg2\" (UniqueName: \"kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.514942 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.514941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home\") pod \"fe497a7d-2574-4d56-81d6-6dcdb7169223\" (UID: \"fe497a7d-2574-4d56-81d6-6dcdb7169223\") "
Jul 07 00:36:22.515199 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515130    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:22.515260 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515183    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:22.515260 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515229    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home" (OuterVolumeSpecName: "home") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:22.515423 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515403    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-tmp-dir\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.515505 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515428    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-model-cache\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.515505 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.515440    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-home\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.516952 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.516922    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm" (OuterVolumeSpecName: "dshm") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:22.517058 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.516976    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2" (OuterVolumeSpecName: "kube-api-access-5pmg2") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "kube-api-access-5pmg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:22.517058 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.516987    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe497a7d-2574-4d56-81d6-6dcdb7169223" (UID: "fe497a7d-2574-4d56-81d6-6dcdb7169223"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:36:22.559195 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.559163    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" path="/var/lib/kubelet/pods/111fabd4-6015-41a0-b0b4-577e3fbd5e9f/volumes"
Jul 07 00:36:22.615984 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.615950    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pmg2\" (UniqueName: \"kubernetes.io/projected/fe497a7d-2574-4d56-81d6-6dcdb7169223-kube-api-access-5pmg2\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.615984 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.615975    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe497a7d-2574-4d56-81d6-6dcdb7169223-dshm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:22.615984 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:22.615989    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe497a7d-2574-4d56-81d6-6dcdb7169223-tls-certs\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:23.264151 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264117    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911068908688"]
Jul 07 00:36:23.264524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264511    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" containerName="main"
Jul 07 00:36:23.264524 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264525    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" containerName="main"
Jul 07 00:36:23.264642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264544    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe497a7d-2574-4d56-81d6-6dcdb7169223" containerName="main"
Jul 07 00:36:23.264642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264549    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe497a7d-2574-4d56-81d6-6dcdb7169223" containerName="main"
Jul 07 00:36:23.264642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264608    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe497a7d-2574-4d56-81d6-6dcdb7169223" containerName="main"
Jul 07 00:36:23.264642 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.264618    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="111fabd4-6015-41a0-b0b4-577e3fbd5e9f" containerName="main"
Jul 07 00:36:23.268132 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.268111    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:23.270939 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.270918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vrnjc\""
Jul 07 00:36:23.273070 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.273044    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908688"]
Jul 07 00:36:23.291665 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.291639    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"
Jul 07 00:36:23.292045 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.291671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c" event={"ID":"fe497a7d-2574-4d56-81d6-6dcdb7169223","Type":"ContainerDied","Data":"d96450734d9ad07ddbf708e516f0aeb6d02769fb4498ff640a6a601e9a811fe9"}
Jul 07 00:36:23.292045 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.291718    2581 scope.go:117] "RemoveContainer" containerID="082f214e304f3756d1d0f88044e911e3198bf60d23b0c3b5e8a93b54d9ae0113"
Jul 07 00:36:23.312702 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.312676    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:36:23.317444 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.317421    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e15ad3/rate-test-model-e15ad3-kserve-9776996cf-b7m5c"]
Jul 07 00:36:23.425209 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.425172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzr7g\" (UniqueName: \"kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g\") pod \"test-curl-7404-139911068908688\" (UID: \"ecf05d57-33a9-4258-bc4b-ecfcc17b972a\") " pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:23.526688 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.526602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzr7g\" (UniqueName: \"kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g\") pod \"test-curl-7404-139911068908688\" (UID: \"ecf05d57-33a9-4258-bc4b-ecfcc17b972a\") " pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:23.535147 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.535118    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzr7g\" (UniqueName: \"kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g\") pod \"test-curl-7404-139911068908688\" (UID: \"ecf05d57-33a9-4258-bc4b-ecfcc17b972a\") " pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:23.579159 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.579132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:23.711937 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.711744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908688"]
Jul 07 00:36:23.714308 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:23.714278    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecf05d57_33a9_4258_bc4b_ecfcc17b972a.slice/crio-a129e1051907c3d4dfbf0b56a4a6c068ced221317ac3186c0b1e86efa12d6544 WatchSource:0}: Error finding container a129e1051907c3d4dfbf0b56a4a6c068ced221317ac3186c0b1e86efa12d6544: Status 404 returned error can't find the container with id a129e1051907c3d4dfbf0b56a4a6c068ced221317ac3186c0b1e86efa12d6544
Jul 07 00:36:23.715636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:23.715612    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:36:24.300931 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:24.300847    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908688" event={"ID":"ecf05d57-33a9-4258-bc4b-ecfcc17b972a","Type":"ContainerStarted","Data":"a129e1051907c3d4dfbf0b56a4a6c068ced221317ac3186c0b1e86efa12d6544"}
Jul 07 00:36:24.558717 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:24.558644    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe497a7d-2574-4d56-81d6-6dcdb7169223" path="/var/lib/kubelet/pods/fe497a7d-2574-4d56-81d6-6dcdb7169223/volumes"
Jul 07 00:36:25.308998 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:25.308962    2581 generic.go:356] "Generic (PLEG): container finished" podID="ecf05d57-33a9-4258-bc4b-ecfcc17b972a" containerID="45f4561b0a654f6595e414fbed656a2b06f07634a73471bb63578f4fae9a4c7f" exitCode=0
Jul 07 00:36:25.309354 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:25.309043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908688" event={"ID":"ecf05d57-33a9-4258-bc4b-ecfcc17b972a","Type":"ContainerDied","Data":"45f4561b0a654f6595e414fbed656a2b06f07634a73471bb63578f4fae9a4c7f"}
Jul 07 00:36:26.446060 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.446034    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:26.467222 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.467198    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911068908688_ecf05d57-33a9-4258-bc4b-ecfcc17b972a/test-curl-7404-139911068908688/0.log"
Jul 07 00:36:26.489610 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.489579    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911068908688"]
Jul 07 00:36:26.494313 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.494289    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911068908688"]
Jul 07 00:36:26.554352 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.554325    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzr7g\" (UniqueName: \"kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g\") pod \"ecf05d57-33a9-4258-bc4b-ecfcc17b972a\" (UID: \"ecf05d57-33a9-4258-bc4b-ecfcc17b972a\") "
Jul 07 00:36:26.556508 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.556480    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g" (OuterVolumeSpecName: "kube-api-access-hzr7g") pod "ecf05d57-33a9-4258-bc4b-ecfcc17b972a" (UID: "ecf05d57-33a9-4258-bc4b-ecfcc17b972a"). InnerVolumeSpecName "kube-api-access-hzr7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:26.559636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.559609    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf05d57-33a9-4258-bc4b-ecfcc17b972a" path="/var/lib/kubelet/pods/ecf05d57-33a9-4258-bc4b-ecfcc17b972a/volumes"
Jul 07 00:36:26.628594 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.628563    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911068909264"]
Jul 07 00:36:26.628987 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.628974    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf05d57-33a9-4258-bc4b-ecfcc17b972a" containerName="test-curl-7404-139911068908688"
Jul 07 00:36:26.629039 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.628989    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf05d57-33a9-4258-bc4b-ecfcc17b972a" containerName="test-curl-7404-139911068908688"
Jul 07 00:36:26.629078 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.629056    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf05d57-33a9-4258-bc4b-ecfcc17b972a" containerName="test-curl-7404-139911068908688"
Jul 07 00:36:26.632308 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.632292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:26.637939 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.637913    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068909264"]
Jul 07 00:36:26.655759 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.655735    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzr7g\" (UniqueName: \"kubernetes.io/projected/ecf05d57-33a9-4258-bc4b-ecfcc17b972a-kube-api-access-hzr7g\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:26.756997 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.756963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkz58\" (UniqueName: \"kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58\") pod \"test-curl-7404-139911068909264\" (UID: \"e1ad8c23-2f0e-4197-918a-0118b5a2c914\") " pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:26.858599 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.858511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkz58\" (UniqueName: \"kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58\") pod \"test-curl-7404-139911068909264\" (UID: \"e1ad8c23-2f0e-4197-918a-0118b5a2c914\") " pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:26.867401 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.867373    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkz58\" (UniqueName: \"kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58\") pod \"test-curl-7404-139911068909264\" (UID: \"e1ad8c23-2f0e-4197-918a-0118b5a2c914\") " pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:26.942958 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:26.942928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:27.073558 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:27.073527    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068909264"]
Jul 07 00:36:27.074322 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:27.074294    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1ad8c23_2f0e_4197_918a_0118b5a2c914.slice/crio-bcbd10a2c514a13b0cab870dce74feab574ab94862019193d3e661289506ce78 WatchSource:0}: Error finding container bcbd10a2c514a13b0cab870dce74feab574ab94862019193d3e661289506ce78: Status 404 returned error can't find the container with id bcbd10a2c514a13b0cab870dce74feab574ab94862019193d3e661289506ce78
Jul 07 00:36:27.321334 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:27.321296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068909264" event={"ID":"e1ad8c23-2f0e-4197-918a-0118b5a2c914","Type":"ContainerStarted","Data":"bcbd10a2c514a13b0cab870dce74feab574ab94862019193d3e661289506ce78"}
Jul 07 00:36:27.322531 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:27.322506    2581 scope.go:117] "RemoveContainer" containerID="45f4561b0a654f6595e414fbed656a2b06f07634a73471bb63578f4fae9a4c7f"
Jul 07 00:36:27.322672 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:27.322503    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908688"
Jul 07 00:36:28.330720 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:28.330683    2581 generic.go:356] "Generic (PLEG): container finished" podID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" containerID="f8f20fb2215c249a3bee464ea9ec732ad2067ad19bde76aad2a47121417e8b86" exitCode=0
Jul 07 00:36:28.331192 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:28.330772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068909264" event={"ID":"e1ad8c23-2f0e-4197-918a-0118b5a2c914","Type":"ContainerDied","Data":"f8f20fb2215c249a3bee464ea9ec732ad2067ad19bde76aad2a47121417e8b86"}
Jul 07 00:36:29.470054 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.470026    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:29.489300 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.489255    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911068909264_e1ad8c23-2f0e-4197-918a-0118b5a2c914/test-curl-7404-139911068909264/0.log"
Jul 07 00:36:29.513670 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.513630    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911068909264"]
Jul 07 00:36:29.517309 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.517286    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911068909264"]
Jul 07 00:36:29.585104 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.585078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkz58\" (UniqueName: \"kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58\") pod \"e1ad8c23-2f0e-4197-918a-0118b5a2c914\" (UID: \"e1ad8c23-2f0e-4197-918a-0118b5a2c914\") "
Jul 07 00:36:29.587011 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.586980    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58" (OuterVolumeSpecName: "kube-api-access-zkz58") pod "e1ad8c23-2f0e-4197-918a-0118b5a2c914" (UID: "e1ad8c23-2f0e-4197-918a-0118b5a2c914"). InnerVolumeSpecName "kube-api-access-zkz58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:29.649104 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.649070    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911068909408"]
Jul 07 00:36:29.649481 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.649466    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" containerName="test-curl-7404-139911068909264"
Jul 07 00:36:29.649526 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.649482    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" containerName="test-curl-7404-139911068909264"
Jul 07 00:36:29.649558 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.649552    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" containerName="test-curl-7404-139911068909264"
Jul 07 00:36:29.652495 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.652478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:29.656574 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.656549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068909408"]
Jul 07 00:36:29.686827 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.686799    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkz58\" (UniqueName: \"kubernetes.io/projected/e1ad8c23-2f0e-4197-918a-0118b5a2c914-kube-api-access-zkz58\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:29.788074 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.788035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzt6\" (UniqueName: \"kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6\") pod \"test-curl-7404-139911068909408\" (UID: \"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9\") " pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:29.889472 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.889384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzt6\" (UniqueName: \"kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6\") pod \"test-curl-7404-139911068909408\" (UID: \"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9\") " pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:29.897477 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.897450    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzt6\" (UniqueName: \"kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6\") pod \"test-curl-7404-139911068909408\" (UID: \"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9\") " pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:29.962614 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:29.962575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:30.091786 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:30.091761    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068909408"]
Jul 07 00:36:30.093979 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:30.093950    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9bf784c_ec47_4cd0_add5_cf24b01c7fa9.slice/crio-3eb1fb844db99d01422e6a8bfb317b2592aba5c61b56a96843c9d1ea672f5d00 WatchSource:0}: Error finding container 3eb1fb844db99d01422e6a8bfb317b2592aba5c61b56a96843c9d1ea672f5d00: Status 404 returned error can't find the container with id 3eb1fb844db99d01422e6a8bfb317b2592aba5c61b56a96843c9d1ea672f5d00
Jul 07 00:36:30.343561 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:30.343532    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:30.343740 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:30.343548    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcbd10a2c514a13b0cab870dce74feab574ab94862019193d3e661289506ce78"
Jul 07 00:36:30.344768 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:30.344733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068909408" event={"ID":"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9","Type":"ContainerStarted","Data":"3eb1fb844db99d01422e6a8bfb317b2592aba5c61b56a96843c9d1ea672f5d00"}
Jul 07 00:36:30.354488 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:30.354463    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911068909264\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" pod="opendatahub/test-curl-7404-139911068909264"
Jul 07 00:36:30.558754 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:30.558718    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ad8c23-2f0e-4197-918a-0118b5a2c914" path="/var/lib/kubelet/pods/e1ad8c23-2f0e-4197-918a-0118b5a2c914/volumes"
Jul 07 00:36:31.351351 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:31.351322    2581 generic.go:356] "Generic (PLEG): container finished" podID="d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" containerID="967d1c9a2b3e6cdaab88569eb905f4dd6257313fda66f3605bc935b3b0284d8b" exitCode=0
Jul 07 00:36:31.351531 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:31.351408    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068909408" event={"ID":"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9","Type":"ContainerDied","Data":"967d1c9a2b3e6cdaab88569eb905f4dd6257313fda66f3605bc935b3b0284d8b"}
Jul 07 00:36:32.494519 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.494496    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:32.514285 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.514260    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911068909408_d9bf784c-ec47-4cd0-add5-cf24b01c7fa9/test-curl-7404-139911068909408/0.log"
Jul 07 00:36:32.535549 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.535521    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911068909408"]
Jul 07 00:36:32.540584 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.540556    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911068909408"]
Jul 07 00:36:32.613459 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.613378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhzt6\" (UniqueName: \"kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6\") pod \"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9\" (UID: \"d9bf784c-ec47-4cd0-add5-cf24b01c7fa9\") "
Jul 07 00:36:32.615374 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.615344    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6" (OuterVolumeSpecName: "kube-api-access-qhzt6") pod "d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" (UID: "d9bf784c-ec47-4cd0-add5-cf24b01c7fa9"). InnerVolumeSpecName "kube-api-access-qhzt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:32.670788 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.670756    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:32.671177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.671163    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" containerName="test-curl-7404-139911068909408"
Jul 07 00:36:32.671223 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.671179    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" containerName="test-curl-7404-139911068909408"
Jul 07 00:36:32.671258 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.671251    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" containerName="test-curl-7404-139911068909408"
Jul 07 00:36:32.674154 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.674137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:32.680488 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.680462    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:32.714428 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.714391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfd52\" (UniqueName: \"kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52\") pod \"test-curl-7404-139911068908832\" (UID: \"cf018e0f-9fc0-4417-b60f-473fc29c5ef9\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:32.714574 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.714538    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhzt6\" (UniqueName: \"kubernetes.io/projected/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9-kube-api-access-qhzt6\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:32.815843 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.815803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfd52\" (UniqueName: \"kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52\") pod \"test-curl-7404-139911068908832\" (UID: \"cf018e0f-9fc0-4417-b60f-473fc29c5ef9\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:32.824124 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.824096    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfd52\" (UniqueName: \"kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52\") pod \"test-curl-7404-139911068908832\" (UID: \"cf018e0f-9fc0-4417-b60f-473fc29c5ef9\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:32.983759 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:32.983724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:33.110112 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:33.110086    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:33.112563 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:33.112534    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf018e0f_9fc0_4417_b60f_473fc29c5ef9.slice/crio-2196cc56cb09208f4059e596aee213b55f267f1589caedfd052fcdeb1aaa8527 WatchSource:0}: Error finding container 2196cc56cb09208f4059e596aee213b55f267f1589caedfd052fcdeb1aaa8527: Status 404 returned error can't find the container with id 2196cc56cb09208f4059e596aee213b55f267f1589caedfd052fcdeb1aaa8527
Jul 07 00:36:33.362408 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:33.362330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908832" event={"ID":"cf018e0f-9fc0-4417-b60f-473fc29c5ef9","Type":"ContainerStarted","Data":"2196cc56cb09208f4059e596aee213b55f267f1589caedfd052fcdeb1aaa8527"}
Jul 07 00:36:33.363481 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:33.363437    2581 scope.go:117] "RemoveContainer" containerID="967d1c9a2b3e6cdaab88569eb905f4dd6257313fda66f3605bc935b3b0284d8b"
Jul 07 00:36:33.363598 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:33.363494    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068909408"
Jul 07 00:36:34.369977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:34.369935    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" containerID="6ec64ce5fb9df7f596e577f7f4716c1a8f2e97f59d6bd35eeed285a4f0ab872c" exitCode=0
Jul 07 00:36:34.369977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:34.369982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908832" event={"ID":"cf018e0f-9fc0-4417-b60f-473fc29c5ef9","Type":"ContainerDied","Data":"6ec64ce5fb9df7f596e577f7f4716c1a8f2e97f59d6bd35eeed285a4f0ab872c"}
Jul 07 00:36:34.557589 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:34.557548    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9bf784c-ec47-4cd0-add5-cf24b01c7fa9" path="/var/lib/kubelet/pods/d9bf784c-ec47-4cd0-add5-cf24b01c7fa9/volumes"
Jul 07 00:36:35.508422 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.508400    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:35.527215 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.527186    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911068908832_cf018e0f-9fc0-4417-b60f-473fc29c5ef9/test-curl-7404-139911068908832/0.log"
Jul 07 00:36:35.542519 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.542491    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfd52\" (UniqueName: \"kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52\") pod \"cf018e0f-9fc0-4417-b60f-473fc29c5ef9\" (UID: \"cf018e0f-9fc0-4417-b60f-473fc29c5ef9\") "
Jul 07 00:36:35.544568 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.544535    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52" (OuterVolumeSpecName: "kube-api-access-dfd52") pod "cf018e0f-9fc0-4417-b60f-473fc29c5ef9" (UID: "cf018e0f-9fc0-4417-b60f-473fc29c5ef9"). InnerVolumeSpecName "kube-api-access-dfd52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:35.551380 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.551358    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:35.554415 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.554392    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:35.644171 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.644091    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfd52\" (UniqueName: \"kubernetes.io/projected/cf018e0f-9fc0-4417-b60f-473fc29c5ef9-kube-api-access-dfd52\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:35.685049 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.685021    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:35.685437 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.685423    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:35.685477 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.685439    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:35.685525 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.685504    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:35.688676 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.688659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:35.695334 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.695308    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:35.745249 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.745221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-978xw\" (UniqueName: \"kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw\") pod \"test-curl-7404-139911068908832\" (UID: \"ac54eea4-1595-4d90-ba62-8aaed523ef3a\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:35.846428 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.846380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-978xw\" (UniqueName: \"kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw\") pod \"test-curl-7404-139911068908832\" (UID: \"ac54eea4-1595-4d90-ba62-8aaed523ef3a\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:35.855040 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.855010    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-978xw\" (UniqueName: \"kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw\") pod \"test-curl-7404-139911068908832\" (UID: \"ac54eea4-1595-4d90-ba62-8aaed523ef3a\") " pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:35.998458 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:35.998423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:36.130574 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.130550    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:36.132722 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:36.132697    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac54eea4_1595_4d90_ba62_8aaed523ef3a.slice/crio-cb43074563d5848872d24eb982a4a8a86dbedb5c05e2a8a17f452224a3fff620 WatchSource:0}: Error finding container cb43074563d5848872d24eb982a4a8a86dbedb5c05e2a8a17f452224a3fff620: Status 404 returned error can't find the container with id cb43074563d5848872d24eb982a4a8a86dbedb5c05e2a8a17f452224a3fff620
Jul 07 00:36:36.382049 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.381971    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2196cc56cb09208f4059e596aee213b55f267f1589caedfd052fcdeb1aaa8527"
Jul 07 00:36:36.382049 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.381971    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:36.383197 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.383163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908832" event={"ID":"ac54eea4-1595-4d90-ba62-8aaed523ef3a","Type":"ContainerStarted","Data":"cb43074563d5848872d24eb982a4a8a86dbedb5c05e2a8a17f452224a3fff620"}
Jul 07 00:36:36.394676 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.394645    2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7404-139911068908832" oldPodUID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a"
Jul 07 00:36:36.418807 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:36.418784    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf018e0f_9fc0_4417_b60f_473fc29c5ef9.slice\": RecentStats: unable to find data in memory cache]"
Jul 07 00:36:36.557731 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:36.557706    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf018e0f-9fc0-4417-b60f-473fc29c5ef9" path="/var/lib/kubelet/pods/cf018e0f-9fc0-4417-b60f-473fc29c5ef9/volumes"
Jul 07 00:36:37.389246 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:37.389209    2581 generic.go:356] "Generic (PLEG): container finished" podID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" containerID="0c8692b26ead129183dfece32ff133121b552cc6491463ad2d745cefdcc730ef" exitCode=0
Jul 07 00:36:37.389425 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:37.389305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911068908832" event={"ID":"ac54eea4-1595-4d90-ba62-8aaed523ef3a","Type":"ContainerDied","Data":"0c8692b26ead129183dfece32ff133121b552cc6491463ad2d745cefdcc730ef"}
Jul 07 00:36:38.533101 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.533076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:38.552735 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.552706    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911068908832_ac54eea4-1595-4d90-ba62-8aaed523ef3a/test-curl-7404-139911068908832/0.log"
Jul 07 00:36:38.571282 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.571255    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-978xw\" (UniqueName: \"kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw\") pod \"ac54eea4-1595-4d90-ba62-8aaed523ef3a\" (UID: \"ac54eea4-1595-4d90-ba62-8aaed523ef3a\") "
Jul 07 00:36:38.573357 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.573319    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw" (OuterVolumeSpecName: "kube-api-access-978xw") pod "ac54eea4-1595-4d90-ba62-8aaed523ef3a" (UID: "ac54eea4-1595-4d90-ba62-8aaed523ef3a"). InnerVolumeSpecName "kube-api-access-978xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:38.575592 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.575569    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:38.579534 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.579510    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911068908832"]
Jul 07 00:36:38.672056 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.671979    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-978xw\" (UniqueName: \"kubernetes.io/projected/ac54eea4-1595-4d90-ba62-8aaed523ef3a-kube-api-access-978xw\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:38.708239 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.708209    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:38.708634 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.708618    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:38.708690 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.708637    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:38.708730 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.708720    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" containerName="test-curl-7404-139911068908832"
Jul 07 00:36:38.711901 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.711882    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:38.718625 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.718601    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:38.773253 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.773222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2rc\" (UniqueName: \"kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc\") pod \"test-curl-7404-139911067867296\" (UID: \"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:38.873954 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.873914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2rc\" (UniqueName: \"kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc\") pod \"test-curl-7404-139911067867296\" (UID: \"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:38.882271 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:38.882239    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2rc\" (UniqueName: \"kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc\") pod \"test-curl-7404-139911067867296\" (UID: \"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:39.023616 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:39.023582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:39.162063 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:39.162034    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:39.164598 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:39.164551    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6c9dec4_7fe1_4d05_bcf7_492fed8a805f.slice/crio-563a6f2b0c8e1d1a9523c87a3f798757875218ef83c5ef14cb67cb7ff97b0fe7 WatchSource:0}: Error finding container 563a6f2b0c8e1d1a9523c87a3f798757875218ef83c5ef14cb67cb7ff97b0fe7: Status 404 returned error can't find the container with id 563a6f2b0c8e1d1a9523c87a3f798757875218ef83c5ef14cb67cb7ff97b0fe7
Jul 07 00:36:39.400918 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:39.400819    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:39.400918 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:39.400889    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb43074563d5848872d24eb982a4a8a86dbedb5c05e2a8a17f452224a3fff620"
Jul 07 00:36:39.402144 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:39.402120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867296" event={"ID":"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f","Type":"ContainerStarted","Data":"563a6f2b0c8e1d1a9523c87a3f798757875218ef83c5ef14cb67cb7ff97b0fe7"}
Jul 07 00:36:39.411504 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:39.411482    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911068908832\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:40.408570 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:40.408534    2581 generic.go:356] "Generic (PLEG): container finished" podID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" containerID="f70176385570f29ff256c6ffe8a50aacaf72f3767f168060ba73cc1cfde9149b" exitCode=0
Jul 07 00:36:40.408982 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:40.408621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867296" event={"ID":"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f","Type":"ContainerDied","Data":"f70176385570f29ff256c6ffe8a50aacaf72f3767f168060ba73cc1cfde9149b"}
Jul 07 00:36:40.411519 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:40.411493    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911068908832\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" pod="opendatahub/test-curl-7404-139911068908832"
Jul 07 00:36:40.557542 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:40.557504    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac54eea4-1595-4d90-ba62-8aaed523ef3a" path="/var/lib/kubelet/pods/ac54eea4-1595-4d90-ba62-8aaed523ef3a/volumes"
Jul 07 00:36:41.572901 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.572873    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:41.591423 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.591392    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911067867296_a6c9dec4-7fe1-4d05-bcf7-492fed8a805f/test-curl-7404-139911067867296/0.log"
Jul 07 00:36:41.597705 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.597680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx2rc\" (UniqueName: \"kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc\") pod \"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f\" (UID: \"a6c9dec4-7fe1-4d05-bcf7-492fed8a805f\") "
Jul 07 00:36:41.599689 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.599667    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc" (OuterVolumeSpecName: "kube-api-access-wx2rc") pod "a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" (UID: "a6c9dec4-7fe1-4d05-bcf7-492fed8a805f"). InnerVolumeSpecName "kube-api-access-wx2rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:41.615391 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.615364    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:41.617710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.617687    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:41.698552 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.698479    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2rc\" (UniqueName: \"kubernetes.io/projected/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f-kube-api-access-wx2rc\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:41.743072 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.743036    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:41.743464 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.743450    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:41.743504 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.743466    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:41.743542 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.743536    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:41.746755 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.746739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:41.753077 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.753053    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:41.799520 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.799488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2cct\" (UniqueName: \"kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct\") pod \"test-curl-7404-139911067867296\" (UID: \"4042c0dd-7d43-4741-ba8f-6cdd166fc99c\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:41.900980 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.900944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2cct\" (UniqueName: \"kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct\") pod \"test-curl-7404-139911067867296\" (UID: \"4042c0dd-7d43-4741-ba8f-6cdd166fc99c\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:41.911571 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:41.911539    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2cct\" (UniqueName: \"kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct\") pod \"test-curl-7404-139911067867296\" (UID: \"4042c0dd-7d43-4741-ba8f-6cdd166fc99c\") " pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:42.056229 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.056195    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:42.184645 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.184616    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:42.186613 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:42.186589    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4042c0dd_7d43_4741_ba8f_6cdd166fc99c.slice/crio-46cff923741b819e180b6b4a7e1a36e0f4e50851d8a492745693561232fa45e7 WatchSource:0}: Error finding container 46cff923741b819e180b6b4a7e1a36e0f4e50851d8a492745693561232fa45e7: Status 404 returned error can't find the container with id 46cff923741b819e180b6b4a7e1a36e0f4e50851d8a492745693561232fa45e7
Jul 07 00:36:42.421693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.421608    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:42.421693 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.421640    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="563a6f2b0c8e1d1a9523c87a3f798757875218ef83c5ef14cb67cb7ff97b0fe7"
Jul 07 00:36:42.422841 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.422821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867296" event={"ID":"4042c0dd-7d43-4741-ba8f-6cdd166fc99c","Type":"ContainerStarted","Data":"46cff923741b819e180b6b4a7e1a36e0f4e50851d8a492745693561232fa45e7"}
Jul 07 00:36:42.433086 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.433061    2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7404-139911067867296" oldPodUID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c"
Jul 07 00:36:42.558407 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:42.558381    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6c9dec4-7fe1-4d05-bcf7-492fed8a805f" path="/var/lib/kubelet/pods/a6c9dec4-7fe1-4d05-bcf7-492fed8a805f/volumes"
Jul 07 00:36:43.429738 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:43.429707    2581 generic.go:356] "Generic (PLEG): container finished" podID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" containerID="7976be444984ab9e486b18a18b401549deba3e88e47ce94410d9eb2091d46656" exitCode=0
Jul 07 00:36:43.430221 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:43.429788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867296" event={"ID":"4042c0dd-7d43-4741-ba8f-6cdd166fc99c","Type":"ContainerDied","Data":"7976be444984ab9e486b18a18b401549deba3e88e47ce94410d9eb2091d46656"}
Jul 07 00:36:44.567828 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.567803    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:44.588161 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.588128    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911067867296_4042c0dd-7d43-4741-ba8f-6cdd166fc99c/test-curl-7404-139911067867296/0.log"
Jul 07 00:36:44.610559 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.610521    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:44.616210 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.616186    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911067867296"]
Jul 07 00:36:44.626477 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.626448    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2cct\" (UniqueName: \"kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct\") pod \"4042c0dd-7d43-4741-ba8f-6cdd166fc99c\" (UID: \"4042c0dd-7d43-4741-ba8f-6cdd166fc99c\") "
Jul 07 00:36:44.628543 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.628522    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct" (OuterVolumeSpecName: "kube-api-access-q2cct") pod "4042c0dd-7d43-4741-ba8f-6cdd166fc99c" (UID: "4042c0dd-7d43-4741-ba8f-6cdd166fc99c"). InnerVolumeSpecName "kube-api-access-q2cct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:44.728143 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.728114    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2cct\" (UniqueName: \"kubernetes.io/projected/4042c0dd-7d43-4741-ba8f-6cdd166fc99c-kube-api-access-q2cct\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:44.752354 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.752319    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911067867440"]
Jul 07 00:36:44.752780 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.752762    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:44.752906 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.752783    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:44.752906 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.752893    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" containerName="test-curl-7404-139911067867296"
Jul 07 00:36:44.755592 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.755574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:44.763109 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.763085    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867440"]
Jul 07 00:36:44.829494 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.829459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2stpc\" (UniqueName: \"kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc\") pod \"test-curl-7404-139911067867440\" (UID: \"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898\") " pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:44.930286 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.930235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2stpc\" (UniqueName: \"kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc\") pod \"test-curl-7404-139911067867440\" (UID: \"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898\") " pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:44.939204 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:44.939179    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2stpc\" (UniqueName: \"kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc\") pod \"test-curl-7404-139911067867440\" (UID: \"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898\") " pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:45.066122 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:45.066025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:45.198184 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:45.198159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067867440"]
Jul 07 00:36:45.200111 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:45.200079    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc248cfdb_ca03_4c42_9fbf_a9eb42f0b898.slice/crio-e98d19259f589efdc6a1e4d8b5841f5cf9df4afcfb4f95d3becb7f3e21ac2846 WatchSource:0}: Error finding container e98d19259f589efdc6a1e4d8b5841f5cf9df4afcfb4f95d3becb7f3e21ac2846: Status 404 returned error can't find the container with id e98d19259f589efdc6a1e4d8b5841f5cf9df4afcfb4f95d3becb7f3e21ac2846
Jul 07 00:36:45.440442 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:45.440352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:45.440442 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:45.440404    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46cff923741b819e180b6b4a7e1a36e0f4e50851d8a492745693561232fa45e7"
Jul 07 00:36:45.441529 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:45.441505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867440" event={"ID":"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898","Type":"ContainerStarted","Data":"e98d19259f589efdc6a1e4d8b5841f5cf9df4afcfb4f95d3becb7f3e21ac2846"}
Jul 07 00:36:45.450965 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:45.450936    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067867296\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:46.447745 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:46.447707    2581 generic.go:356] "Generic (PLEG): container finished" podID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" containerID="040254ab2b80db057c79c8292d5047ffce6da206b9e09140cd848a3d91fb9217" exitCode=0
Jul 07 00:36:46.448134 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:46.447793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067867440" event={"ID":"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898","Type":"ContainerDied","Data":"040254ab2b80db057c79c8292d5047ffce6da206b9e09140cd848a3d91fb9217"}
Jul 07 00:36:46.450452 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:46.450428    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067867296\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" pod="opendatahub/test-curl-7404-139911067867296"
Jul 07 00:36:46.558375 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:46.558339    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4042c0dd-7d43-4741-ba8f-6cdd166fc99c" path="/var/lib/kubelet/pods/4042c0dd-7d43-4741-ba8f-6cdd166fc99c/volumes"
Jul 07 00:36:47.591141 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.591117    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:47.609908 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.609847    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911067867440_c248cfdb-ca03-4c42-9fbf-a9eb42f0b898/test-curl-7404-139911067867440/0.log"
Jul 07 00:36:47.632806 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.632770    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911067867440"]
Jul 07 00:36:47.636653 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.636629    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911067867440"]
Jul 07 00:36:47.653920 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.653893    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2stpc\" (UniqueName: \"kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc\") pod \"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898\" (UID: \"c248cfdb-ca03-4c42-9fbf-a9eb42f0b898\") "
Jul 07 00:36:47.655977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.655954    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc" (OuterVolumeSpecName: "kube-api-access-2stpc") pod "c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" (UID: "c248cfdb-ca03-4c42-9fbf-a9eb42f0b898"). InnerVolumeSpecName "kube-api-access-2stpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:47.754988 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.754951    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2stpc\" (UniqueName: \"kubernetes.io/projected/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898-kube-api-access-2stpc\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:47.762120 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.762092    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911067865712"]
Jul 07 00:36:47.762491 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.762478    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" containerName="test-curl-7404-139911067867440"
Jul 07 00:36:47.762535 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.762493    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" containerName="test-curl-7404-139911067867440"
Jul 07 00:36:47.762604 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.762595    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" containerName="test-curl-7404-139911067867440"
Jul 07 00:36:47.765533 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.765516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:47.769986 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.769952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067865712"]
Jul 07 00:36:47.855654 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.855620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lfjd\" (UniqueName: \"kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd\") pod \"test-curl-7404-139911067865712\" (UID: \"32e90c72-f823-435d-9e34-2b45d8d10956\") " pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:47.956967 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.956928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lfjd\" (UniqueName: \"kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd\") pod \"test-curl-7404-139911067865712\" (UID: \"32e90c72-f823-435d-9e34-2b45d8d10956\") " pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:47.966178 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:47.966147    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lfjd\" (UniqueName: \"kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd\") pod \"test-curl-7404-139911067865712\" (UID: \"32e90c72-f823-435d-9e34-2b45d8d10956\") " pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:48.075538 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.075447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:48.206929 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.206907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911067865712"]
Jul 07 00:36:48.208942 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:36:48.208915    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32e90c72_f823_435d_9e34_2b45d8d10956.slice/crio-054dd4e0e80d4ee459d6c844c6c3d80e590d5430ec1572fbb4212961d7ff52ff WatchSource:0}: Error finding container 054dd4e0e80d4ee459d6c844c6c3d80e590d5430ec1572fbb4212961d7ff52ff: Status 404 returned error can't find the container with id 054dd4e0e80d4ee459d6c844c6c3d80e590d5430ec1572fbb4212961d7ff52ff
Jul 07 00:36:48.460403 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.460312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067865712" event={"ID":"32e90c72-f823-435d-9e34-2b45d8d10956","Type":"ContainerStarted","Data":"054dd4e0e80d4ee459d6c844c6c3d80e590d5430ec1572fbb4212961d7ff52ff"}
Jul 07 00:36:48.461681 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.461659    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e98d19259f589efdc6a1e4d8b5841f5cf9df4afcfb4f95d3becb7f3e21ac2846"
Jul 07 00:36:48.461681 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.461679    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:48.471885 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:48.471837    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067867440\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:48.559347 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:48.559310    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067867440\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" pod="opendatahub/test-curl-7404-139911067867440"
Jul 07 00:36:48.559895 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:48.559847    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c248cfdb-ca03-4c42-9fbf-a9eb42f0b898" path="/var/lib/kubelet/pods/c248cfdb-ca03-4c42-9fbf-a9eb42f0b898/volumes"
Jul 07 00:36:49.470948 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:49.470914    2581 generic.go:356] "Generic (PLEG): container finished" podID="32e90c72-f823-435d-9e34-2b45d8d10956" containerID="ec7961b4465b7517ec5bbe0891342a0cd137855a8ea3bb49643cacad90f2901e" exitCode=0
Jul 07 00:36:49.471354 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:49.470983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911067865712" event={"ID":"32e90c72-f823-435d-9e34-2b45d8d10956","Type":"ContainerDied","Data":"ec7961b4465b7517ec5bbe0891342a0cd137855a8ea3bb49643cacad90f2901e"}
Jul 07 00:36:50.612401 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.612374    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:50.634147 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.634120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911067865712_32e90c72-f823-435d-9e34-2b45d8d10956/test-curl-7404-139911067865712/0.log"
Jul 07 00:36:50.657249 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.657216    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911067865712"]
Jul 07 00:36:50.662277 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.662251    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911067865712"]
Jul 07 00:36:50.681520 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.681489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lfjd\" (UniqueName: \"kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd\") pod \"32e90c72-f823-435d-9e34-2b45d8d10956\" (UID: \"32e90c72-f823-435d-9e34-2b45d8d10956\") "
Jul 07 00:36:50.683537 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.683508    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd" (OuterVolumeSpecName: "kube-api-access-7lfjd") pod "32e90c72-f823-435d-9e34-2b45d8d10956" (UID: "32e90c72-f823-435d-9e34-2b45d8d10956"). InnerVolumeSpecName "kube-api-access-7lfjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:50.783143 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:50.783052    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lfjd\" (UniqueName: \"kubernetes.io/projected/32e90c72-f823-435d-9e34-2b45d8d10956-kube-api-access-7lfjd\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:51.075266 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.075188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:51.485741 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.485702    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:51.488166 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.488146    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="054dd4e0e80d4ee459d6c844c6c3d80e590d5430ec1572fbb4212961d7ff52ff"
Jul 07 00:36:51.488281 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.488179    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:51.497890 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:51.497838    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067865712\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:51.579153 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.579117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:51.680496 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.680457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:51.719636 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.719604    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:36:51.719869 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.719829    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" podUID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" containerName="maas-api" containerID="cri-o://b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27" gracePeriod=30
Jul 07 00:36:51.739066 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:51.738980    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067865712\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:51.781369 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:51.781330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:52.178585 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.178562    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:36:52.198395 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:52.198355    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067865712\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:52.298755 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.298668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") pod \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") "
Jul 07 00:36:52.298755 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.298727    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca\") pod \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") "
Jul 07 00:36:52.298992 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.298842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xhfm\" (UniqueName: \"kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm\") pod \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\" (UID: \"0e9d2101-ff79-41ed-b75d-6358b7c48bce\") "
Jul 07 00:36:52.299106 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.299083    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e9d2101-ff79-41ed-b75d-6358b7c48bce" (UID: "0e9d2101-ff79-41ed-b75d-6358b7c48bce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:36:52.299176 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.299161    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9d2101-ff79-41ed-b75d-6358b7c48bce-openshift-service-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:52.300816 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.300796    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e9d2101-ff79-41ed-b75d-6358b7c48bce" (UID: "0e9d2101-ff79-41ed-b75d-6358b7c48bce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:36:52.300957 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.300941    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm" (OuterVolumeSpecName: "kube-api-access-7xhfm") pod "0e9d2101-ff79-41ed-b75d-6358b7c48bce" (UID: "0e9d2101-ff79-41ed-b75d-6358b7c48bce"). InnerVolumeSpecName "kube-api-access-7xhfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:52.400291 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.400257    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xhfm\" (UniqueName: \"kubernetes.io/projected/0e9d2101-ff79-41ed-b75d-6358b7c48bce-kube-api-access-7xhfm\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:52.400291 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.400286    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9d2101-ff79-41ed-b75d-6358b7c48bce-maas-api-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:52.494934 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.494898    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" containerID="b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27" exitCode=0
Jul 07 00:36:52.495102 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.494944    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"
Jul 07 00:36:52.495102 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.494989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" event={"ID":"0e9d2101-ff79-41ed-b75d-6358b7c48bce","Type":"ContainerDied","Data":"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"}
Jul 07 00:36:52.495102 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.495043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58" event={"ID":"0e9d2101-ff79-41ed-b75d-6358b7c48bce","Type":"ContainerDied","Data":"44bf53190aca625c7e91a1278cc33d5ab8821520883d1bc7394b3390cd9ae7c3"}
Jul 07 00:36:52.495102 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.495067    2581 scope.go:117] "RemoveContainer" containerID="b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"
Jul 07 00:36:52.501152 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:52.501121    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067865712\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:52.504665 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.504646    2581 scope.go:117] "RemoveContainer" containerID="b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"
Jul 07 00:36:52.504954 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:52.504931    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27\": container with ID starting with b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27 not found: ID does not exist" containerID="b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"
Jul 07 00:36:52.505001 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.504963    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27"} err="failed to get container status \"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27\": rpc error: code = NotFound desc = could not find container \"b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27\": container with ID starting with b80aa23e5834e954cbf812ca63fa5babfacb34f0e435bf02bb51ead540c51c27 not found: ID does not exist"
Jul 07 00:36:52.516570 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.516549    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:36:52.519553 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.519532    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ea3965-58f7f89db5-h8j58"]
Jul 07 00:36:52.519644 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:52.519553    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911067865712\" is forbidden: User \"system:node:ip-10-0-129-135.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-135.ec2.internal' and this object" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" pod="opendatahub/test-curl-7404-139911067865712"
Jul 07 00:36:52.557853 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.557797    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" path="/var/lib/kubelet/pods/0e9d2101-ff79-41ed-b75d-6358b7c48bce/volumes"
Jul 07 00:36:52.558232 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:52.558217    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" path="/var/lib/kubelet/pods/32e90c72-f823-435d-9e34-2b45d8d10956/volumes"
Jul 07 00:36:57.374865 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:57.374828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:57.437852 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:57.437803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:57.645944 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:57.645847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:57.886201 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:57.886171    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:58.068424 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.068376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:58.579451 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.579416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:58.678264 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.678229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:58.778900 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.778852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:58.832433 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.832355    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:36:58.832616 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.832591    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" podUID="f9757c2f-58a3-46ce-b894-cac099990602" containerName="maas-api" containerID="cri-o://17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea" gracePeriod=30
Jul 07 00:36:58.879371 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:58.879341    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:59.273710 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.273684    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:36:59.361006 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.360911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvj6c\" (UniqueName: \"kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c\") pod \"f9757c2f-58a3-46ce-b894-cac099990602\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") "
Jul 07 00:36:59.361006 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.360953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") pod \"f9757c2f-58a3-46ce-b894-cac099990602\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") "
Jul 07 00:36:59.361006 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.361000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca\") pod \"f9757c2f-58a3-46ce-b894-cac099990602\" (UID: \"f9757c2f-58a3-46ce-b894-cac099990602\") "
Jul 07 00:36:59.361409 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.361379    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9757c2f-58a3-46ce-b894-cac099990602" (UID: "f9757c2f-58a3-46ce-b894-cac099990602"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:36:59.363178 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.363151    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9757c2f-58a3-46ce-b894-cac099990602" (UID: "f9757c2f-58a3-46ce-b894-cac099990602"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:36:59.363278 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.363156    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c" (OuterVolumeSpecName: "kube-api-access-xvj6c") pod "f9757c2f-58a3-46ce-b894-cac099990602" (UID: "f9757c2f-58a3-46ce-b894-cac099990602"). InnerVolumeSpecName "kube-api-access-xvj6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:59.462158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.462123    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvj6c\" (UniqueName: \"kubernetes.io/projected/f9757c2f-58a3-46ce-b894-cac099990602-kube-api-access-xvj6c\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:59.462158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.462154    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9757c2f-58a3-46ce-b894-cac099990602-maas-api-tls\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:59.462158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.462164    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9757c2f-58a3-46ce-b894-cac099990602-openshift-service-ca\") on node \"ip-10-0-129-135.ec2.internal\" DevicePath \"\""
Jul 07 00:36:59.533768 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.533729    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9757c2f-58a3-46ce-b894-cac099990602" containerID="17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea" exitCode=0
Jul 07 00:36:59.533960 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.533777    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"
Jul 07 00:36:59.533960 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.533789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" event={"ID":"f9757c2f-58a3-46ce-b894-cac099990602","Type":"ContainerDied","Data":"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"}
Jul 07 00:36:59.533960 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.533842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms" event={"ID":"f9757c2f-58a3-46ce-b894-cac099990602","Type":"ContainerDied","Data":"83bfe1e0d3c11ba0dba5e1fd82966755c8a7c73912228d245f20c4405bb1ae44"}
Jul 07 00:36:59.533960 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.533892    2581 scope.go:117] "RemoveContainer" containerID="17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"
Jul 07 00:36:59.543450 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.543430    2581 scope.go:117] "RemoveContainer" containerID="17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"
Jul 07 00:36:59.543701 ip-10-0-129-135 kubenswrapper[2581]: E0707 00:36:59.543682    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea\": container with ID starting with 17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea not found: ID does not exist" containerID="17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"
Jul 07 00:36:59.543778 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.543710    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea"} err="failed to get container status \"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea\": rpc error: code = NotFound desc = could not find container \"17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea\": container with ID starting with 17f8bfeef30f1dbabf3a942716b053267cff0817768cf37dcdd487db8219f1ea not found: ID does not exist"
Jul 07 00:36:59.555512 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.555481    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:36:59.559164 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:36:59.559144    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-e15ad3-6bdfc6bb6b-v9nms"]
Jul 07 00:37:00.558024 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:00.557930    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9757c2f-58a3-46ce-b894-cac099990602" path="/var/lib/kubelet/pods/f9757c2f-58a3-46ce-b894-cac099990602/volumes"
Jul 07 00:37:04.335614 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:04.335581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:04.436342 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:04.436302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:04.625252 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:04.625151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:04.856934 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:04.856903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:10.399678 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:10.399640    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-4pjbr_70fb7991-9c90-4023-a749-49412823b15a/cleanup/0.log"
Jul 07 00:37:10.527705 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:10.527676    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723070-jpgdq_2f890ca2-7318-4e4c-be19-4914dbae9fde/cleanup/0.log"
Jul 07 00:37:11.033208 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:11.033182    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-8tdd7_454c713c-b8cf-4016-8318-7d292e93a31a/copy-manifests/0.log"
Jul 07 00:37:11.046066 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:11.046030    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-8tdd7_454c713c-b8cf-4016-8318-7d292e93a31a/manager/0.log"
Jul 07 00:37:12.994497 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:12.994465    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6jlnh_2dac88b5-a509-4c69-9da5-f9aced210220/manager/0.log"
Jul 07 00:37:13.103567 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:13.103545    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z4drd_4846c3e9-9731-4323-a74c-79208577f305/kuadrant-console-plugin/0.log"
Jul 07 00:37:13.354726 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:13.354651    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n5ph5_8fab5965-cde9-4f7c-b450-9b95afb1ae93/manager/0.log"
Jul 07 00:37:14.239266 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:14.239229    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-rh64d_2b6c3f8e-6c09-4d6b-8b32-3df74058ede5/kube-auth-proxy/0.log"
Jul 07 00:37:14.625484 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:14.625456    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t7tvs_9208b2e8-54bc-4319-8b94-fd9f6d3bb37c/payload-pre-processing/0.log"
Jul 07 00:37:15.355509 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:15.355478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wl8bw_a3578d5d-465d-4ee0-9f1b-8c3d10e60a2e/main/0.log"
Jul 07 00:37:15.597543 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:15.597514    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2cwzr_0f6367df-4d2f-4121-a3bf-def991e0033b/main/0.log"
Jul 07 00:37:20.439470 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.439429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:20.439938 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.439876    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vhskf/must-gather-498wt"]
Jul 07 00:37:20.440261 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440248    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9757c2f-58a3-46ce-b894-cac099990602" containerName="maas-api"
Jul 07 00:37:20.440298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440262    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9757c2f-58a3-46ce-b894-cac099990602" containerName="maas-api"
Jul 07 00:37:20.440298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440273    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" containerName="maas-api"
Jul 07 00:37:20.440298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440280    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" containerName="maas-api"
Jul 07 00:37:20.440298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440290    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" containerName="test-curl-7404-139911067865712"
Jul 07 00:37:20.440298 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440296    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" containerName="test-curl-7404-139911067865712"
Jul 07 00:37:20.440454 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440355    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9757c2f-58a3-46ce-b894-cac099990602" containerName="maas-api"
Jul 07 00:37:20.440454 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440363    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e90c72-f823-435d-9e34-2b45d8d10956" containerName="test-curl-7404-139911067865712"
Jul 07 00:37:20.440454 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.440373    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9d2101-ff79-41ed-b75d-6358b7c48bce" containerName="maas-api"
Jul 07 00:37:20.443595 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.443581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.448167 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.448148    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vhskf\"/\"openshift-service-ca.crt\""
Jul 07 00:37:20.448286 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.448166    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vhskf\"/\"default-dockercfg-hdd62\""
Jul 07 00:37:20.449197 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.449180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vhskf\"/\"kube-root-ca.crt\""
Jul 07 00:37:20.453727 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.453705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/must-gather-498wt"]
Jul 07 00:37:20.458364 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.458338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33b3feef-83df-4cb7-8468-9efa761bb225-must-gather-output\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.458445 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.458384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph6rn\" (UniqueName: \"kubernetes.io/projected/33b3feef-83df-4cb7-8468-9efa761bb225-kube-api-access-ph6rn\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.537422 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.537394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:20.559046 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.559019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33b3feef-83df-4cb7-8468-9efa761bb225-must-gather-output\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.559206 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.559052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph6rn\" (UniqueName: \"kubernetes.io/projected/33b3feef-83df-4cb7-8468-9efa761bb225-kube-api-access-ph6rn\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.559407 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.559388    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33b3feef-83df-4cb7-8468-9efa761bb225-must-gather-output\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.569424 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.569400    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph6rn\" (UniqueName: \"kubernetes.io/projected/33b3feef-83df-4cb7-8468-9efa761bb225-kube-api-access-ph6rn\") pod \"must-gather-498wt\" (UID: \"33b3feef-83df-4cb7-8468-9efa761bb225\") " pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.753094 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.753057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/must-gather-498wt"
Jul 07 00:37:20.886956 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:20.886926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/must-gather-498wt"]
Jul 07 00:37:20.888756 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:37:20.888730    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33b3feef_83df_4cb7_8468_9efa761bb225.slice/crio-e9d79d149f67b468c99ad602ed6c15e96487e00b233c810a6837db6d34bbf1d3 WatchSource:0}: Error finding container e9d79d149f67b468c99ad602ed6c15e96487e00b233c810a6837db6d34bbf1d3: Status 404 returned error can't find the container with id e9d79d149f67b468c99ad602ed6c15e96487e00b233c810a6837db6d34bbf1d3
Jul 07 00:37:21.648541 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:21.648508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/must-gather-498wt" event={"ID":"33b3feef-83df-4cb7-8468-9efa761bb225","Type":"ContainerStarted","Data":"e9d79d149f67b468c99ad602ed6c15e96487e00b233c810a6837db6d34bbf1d3"}
Jul 07 00:37:22.658732 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:22.658690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/must-gather-498wt" event={"ID":"33b3feef-83df-4cb7-8468-9efa761bb225","Type":"ContainerStarted","Data":"2d4a3d74ca5074cc9f21fabf7eea8939cf990c1d0295501bb8a363152dc1b78c"}
Jul 07 00:37:22.659177 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:22.658749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/must-gather-498wt" event={"ID":"33b3feef-83df-4cb7-8468-9efa761bb225","Type":"ContainerStarted","Data":"2566a8268a0f4df1537f5bf0d9b2528c23d3abbc6170c70c677c96c3427c76a8"}
Jul 07 00:37:22.676236 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:22.676175    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vhskf/must-gather-498wt" podStartSLOduration=2.017511509 podStartE2EDuration="2.676153507s" totalImagesPullingTime="658.641998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:37:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:37:22.675208426 +0000 UTC m=+2445.078604500" watchObservedRunningTime="2026-07-07 00:37:22.676153507 +0000 UTC m=+2445.079549580"
Jul 07 00:37:23.123687 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:23.123658    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hp5lp_5b8ea7e7-5219-4e2b-a7ee-cdf90cd72c63/global-pull-secret-syncer/0.log"
Jul 07 00:37:23.217532 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:23.217504    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hf98m_6e867e8f-9177-4f92-a6c7-e64255669905/konnectivity-agent/0.log"
Jul 07 00:37:23.346599 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:23.346572    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-135.ec2.internal_200bd357d82b41d3210c766efd1f8f3e/haproxy/0.log"
Jul 07 00:37:28.882012 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:28.881928    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-6jlnh_2dac88b5-a509-4c69-9da5-f9aced210220/manager/0.log"
Jul 07 00:37:28.909816 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:28.909771    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z4drd_4846c3e9-9731-4323-a74c-79208577f305/kuadrant-console-plugin/0.log"
Jul 07 00:37:29.116101 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:29.116057    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n5ph5_8fab5965-cde9-4f7c-b450-9b95afb1ae93/manager/0.log"
Jul 07 00:37:30.937227 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:30.937195    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/alertmanager/0.log"
Jul 07 00:37:30.959979 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:30.959950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/config-reloader/0.log"
Jul 07 00:37:30.982077 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:30.982030    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/kube-rbac-proxy-web/0.log"
Jul 07 00:37:31.003897 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.003803    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/kube-rbac-proxy/0.log"
Jul 07 00:37:31.027635 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.027549    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/kube-rbac-proxy-metric/0.log"
Jul 07 00:37:31.056220 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.056196    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/prom-label-proxy/0.log"
Jul 07 00:37:31.081376 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.081328    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_65644a99-09ac-4b14-9fa3-c1d786556999/init-config-reloader/0.log"
Jul 07 00:37:31.158324 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.158276    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-stvj5_ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1/kube-state-metrics/0.log"
Jul 07 00:37:31.178005 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.177975    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-stvj5_ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1/kube-rbac-proxy-main/0.log"
Jul 07 00:37:31.202604 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.202566    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-stvj5_ba5355cc-fbfe-45c6-b4d6-92e7a7c039f1/kube-rbac-proxy-self/0.log"
Jul 07 00:37:31.229065 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.229032    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86c58d4676-7phlj_012ead98-7d13-4ef0-8863-5ea867e01fca/metrics-server/0.log"
Jul 07 00:37:31.266334 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.266252    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-wgzs6_7776f20b-0ce6-47fa-9229-06ceb3ba3130/monitoring-plugin/0.log"
Jul 07 00:37:31.471360 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.471335    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5gzj_48b7d031-87ca-4fe0-b7ec-ae5554d05a7c/node-exporter/0.log"
Jul 07 00:37:31.493401 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.493368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5gzj_48b7d031-87ca-4fe0-b7ec-ae5554d05a7c/kube-rbac-proxy/0.log"
Jul 07 00:37:31.517647 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.517615    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5gzj_48b7d031-87ca-4fe0-b7ec-ae5554d05a7c/init-textfile/0.log"
Jul 07 00:37:31.546235 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.546209    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4l7rc_1773a6ee-bff4-404c-9bbc-ad641b078b9a/kube-rbac-proxy-main/0.log"
Jul 07 00:37:31.569349 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.569319    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4l7rc_1773a6ee-bff4-404c-9bbc-ad641b078b9a/kube-rbac-proxy-self/0.log"
Jul 07 00:37:31.595809 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.595780    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-4l7rc_1773a6ee-bff4-404c-9bbc-ad641b078b9a/openshift-state-metrics/0.log"
Jul 07 00:37:31.635420 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.635388    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/prometheus/0.log"
Jul 07 00:37:31.658650 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.658624    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/config-reloader/0.log"
Jul 07 00:37:31.682663 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.682629    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/thanos-sidecar/0.log"
Jul 07 00:37:31.705158 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.705130    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/kube-rbac-proxy-web/0.log"
Jul 07 00:37:31.729516 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.729443    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/kube-rbac-proxy/0.log"
Jul 07 00:37:31.748758 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.748727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/kube-rbac-proxy-thanos/0.log"
Jul 07 00:37:31.770567 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.770537    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5bf10432-8644-4a6c-a0ef-3708bee2e33a/init-config-reloader/0.log"
Jul 07 00:37:31.801230 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.801199    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-8lspz_86651ac1-28cc-4309-bc1b-436402a8fa72/prometheus-operator/0.log"
Jul 07 00:37:31.819673 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.819638    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-8lspz_86651ac1-28cc-4309-bc1b-436402a8fa72/kube-rbac-proxy/0.log"
Jul 07 00:37:31.878788 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.878754    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85db4fc479-lv25d_bf496a13-facb-4e4d-8f0d-b4aa7b69eadd/telemeter-client/0.log"
Jul 07 00:37:31.902211 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.902173    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85db4fc479-lv25d_bf496a13-facb-4e4d-8f0d-b4aa7b69eadd/reload/0.log"
Jul 07 00:37:31.943382 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.943347    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85db4fc479-lv25d_bf496a13-facb-4e4d-8f0d-b4aa7b69eadd/kube-rbac-proxy/0.log"
Jul 07 00:37:31.970697 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.970668    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/thanos-query/0.log"
Jul 07 00:37:31.992506 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:31.992403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/kube-rbac-proxy-web/0.log"
Jul 07 00:37:32.016917 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:32.016886    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/kube-rbac-proxy/0.log"
Jul 07 00:37:32.037254 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:32.037220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/prom-label-proxy/0.log"
Jul 07 00:37:32.068832 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:32.068804    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/kube-rbac-proxy-rules/0.log"
Jul 07 00:37:32.095419 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:32.095362    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7788cc8d99-llz8p_19e1111f-023f-4214-a572-941c0ed562e1/kube-rbac-proxy-metrics/0.log"
Jul 07 00:37:33.259958 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.259920    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"]
Jul 07 00:37:33.268404 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.268379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.277563 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.277529    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"]
Jul 07 00:37:33.410926 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.410880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-sys\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.411120 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.410970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-podres\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.411120 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.411056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-lib-modules\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.411120 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.411085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwbnz\" (UniqueName: \"kubernetes.io/projected/7bc6728e-1c85-4c58-800c-c79368fff1bb-kube-api-access-kwbnz\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.411292 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.411125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-proc\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512723 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-lib-modules\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512723 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwbnz\" (UniqueName: \"kubernetes.io/projected/7bc6728e-1c85-4c58-800c-c79368fff1bb-kube-api-access-kwbnz\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-proc\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-sys\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-podres\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512927    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-proc\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.512989 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.512839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-lib-modules\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.513366 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.513031    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-sys\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.513366 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.513122    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bc6728e-1c85-4c58-800c-c79368fff1bb-podres\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.521586 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.521550    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwbnz\" (UniqueName: \"kubernetes.io/projected/7bc6728e-1c85-4c58-800c-c79368fff1bb-kube-api-access-kwbnz\") pod \"perf-node-gather-daemonset-crtzt\" (UID: \"7bc6728e-1c85-4c58-800c-c79368fff1bb\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.590393 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.590351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:33.794070 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:33.794037    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"]
Jul 07 00:37:33.796420 ip-10-0-129-135 kubenswrapper[2581]: W0707 00:37:33.796390    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7bc6728e_1c85_4c58_800c_c79368fff1bb.slice/crio-558d8383e4493da970c59b66bf1383b78e815520957a6009a124d827f8dd69d8 WatchSource:0}: Error finding container 558d8383e4493da970c59b66bf1383b78e815520957a6009a124d827f8dd69d8: Status 404 returned error can't find the container with id 558d8383e4493da970c59b66bf1383b78e815520957a6009a124d827f8dd69d8
Jul 07 00:37:34.757350 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:34.757307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt" event={"ID":"7bc6728e-1c85-4c58-800c-c79368fff1bb","Type":"ContainerStarted","Data":"61e1e54aa1c8cad8d6be3c0fd793606b9838a1767cf9953dfd2715d82dfe4848"}
Jul 07 00:37:34.757830 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:34.757389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt" event={"ID":"7bc6728e-1c85-4c58-800c-c79368fff1bb","Type":"ContainerStarted","Data":"558d8383e4493da970c59b66bf1383b78e815520957a6009a124d827f8dd69d8"}
Jul 07 00:37:34.757830 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:34.757630    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:34.774055 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:34.773962    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt" podStartSLOduration=1.773939021 podStartE2EDuration="1.773939021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:37:34.773704281 +0000 UTC m=+2457.177100355" watchObservedRunningTime="2026-07-07 00:37:34.773939021 +0000 UTC m=+2457.177335095"
Jul 07 00:37:35.979887 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:35.979844    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7tsxx_5a212e00-f6d3-41cc-8cf0-63bfa3dc9560/dns/0.log"
Jul 07 00:37:35.999890 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:35.999864    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7tsxx_5a212e00-f6d3-41cc-8cf0-63bfa3dc9560/kube-rbac-proxy/0.log"
Jul 07 00:37:36.087415 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:36.087390    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ls9cb_a1ac883a-77ea-4b65-bc6f-9b184e9a5be0/dns-node-resolver/0.log"
Jul 07 00:37:36.707031 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:36.707000    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxqwg_914c2087-5c4e-4e81-a3eb-f573745db5b4/node-ca/0.log"
Jul 07 00:37:37.885308 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:37.885280    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-rh64d_2b6c3f8e-6c09-4d6b-8b32-3df74058ede5/kube-auth-proxy/0.log"
Jul 07 00:37:38.040786 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:38.040756    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t7tvs_9208b2e8-54bc-4319-8b94-fd9f6d3bb37c/payload-pre-processing/0.log"
Jul 07 00:37:38.843773 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:38.843738    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7j9lz_44018009-fed8-42aa-944b-3095c06fa355/serve-healthcheck-canary/0.log"
Jul 07 00:37:39.458358 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:39.458329    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pvsp8_edb12a13-576a-4e8c-9b51-4ecf16debbf4/kube-rbac-proxy/0.log"
Jul 07 00:37:39.480663 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:39.480630    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pvsp8_edb12a13-576a-4e8c-9b51-4ecf16debbf4/exporter/0.log"
Jul 07 00:37:39.501372 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:39.501340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pvsp8_edb12a13-576a-4e8c-9b51-4ecf16debbf4/extractor/0.log"
Jul 07 00:37:40.775300 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:40.775272    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-crtzt"
Jul 07 00:37:42.091328 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:42.091256    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-4pjbr_70fb7991-9c90-4023-a749-49412823b15a/cleanup/0.log"
Jul 07 00:37:42.126690 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:42.126654    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723070-jpgdq_2f890ca2-7318-4e4c-be19-4914dbae9fde/cleanup/0.log"
Jul 07 00:37:42.509386 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:42.509344    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-8tdd7_454c713c-b8cf-4016-8318-7d292e93a31a/manager/0.log"
Jul 07 00:37:42.528977 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:42.528950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-8tdd7_454c713c-b8cf-4016-8318-7d292e93a31a/copy-manifests/0.log"
Jul 07 00:37:44.029945 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:44.029914    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-rh64d_2b6c3f8e-6c09-4d6b-8b32-3df74058ede5/kube-auth-proxy/0.log"
Jul 07 00:37:44.040522 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:44.040497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c45554c7d-kbbxp_400b3e77-a624-4844-9f5d-8c658a9fa003/manager/0.log"
Jul 07 00:37:44.172843 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:44.172810    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t7tvs_9208b2e8-54bc-4319-8b94-fd9f6d3bb37c/payload-pre-processing/0.log"
Jul 07 00:37:45.641438 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:45.641409    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-rh64d_2b6c3f8e-6c09-4d6b-8b32-3df74058ede5/kube-auth-proxy/0.log"
Jul 07 00:37:45.776971 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:45.776940    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t7tvs_9208b2e8-54bc-4319-8b94-fd9f6d3bb37c/payload-pre-processing/0.log"
Jul 07 00:37:50.439948 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.439919    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/kube-multus-additional-cni-plugins/0.log"
Jul 07 00:37:50.460734 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.460709    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/egress-router-binary-copy/0.log"
Jul 07 00:37:50.486114 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.486073    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/cni-plugins/0.log"
Jul 07 00:37:50.507271 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.507247    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/bond-cni-plugin/0.log"
Jul 07 00:37:50.528154 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.528129    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/routeoverride-cni/0.log"
Jul 07 00:37:50.548480 ip-10-0-129-135 kubenswrapper[2581]: I0707 00:37:50.548449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dpx2n_51e3d0e3-d802-42cb-89e8-29aed5331d2f/whereabouts-cni-bincopy/0.log"