Jul 29 20:45:26.405586 ip-10-0-135-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 20:45:26.405592 ip-10-0-135-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 20:45:26.405601 ip-10-0-135-229 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 20:45:26.405901 ip-10-0-135-229 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 20:45:36.644709 ip-10-0-135-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 20:45:36.644735 ip-10-0-135-229 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9b1cd9d81bb24fdabbcafa7f2d61c56f --
Jul 29 20:47:29.637672 ip-10-0-135-229 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 20:47:30.032090 ip-10-0-135-229 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:47:30.032090 ip-10-0-135-229 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 20:47:30.032090 ip-10-0-135-229 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:47:30.032090 ip-10-0-135-229 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 20:47:30.032090 ip-10-0-135-229 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:47:30.032784 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.032155    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 20:47:30.036117 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036099    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.036117 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036117    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036121    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036124    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036127    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036129    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036132    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036135    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036138    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036140    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036143    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036145    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036148    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036150    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036153    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036155    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036158    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036160    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036162    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036166    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036168    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.036230 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036171    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036174    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036178    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036180    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036182    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036185    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036188    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036190    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036192    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036195    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036200    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036203    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036206    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036208    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036212    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036215    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036217    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036220    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036222    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.036706 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036224    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036228    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036231    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036233    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036235    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036238    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036240    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036243    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036245    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036248    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036252    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036255    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036258    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036260    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036263    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036266    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036268    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036271    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036274    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036276    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.037151 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036279    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036282    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036285    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036288    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036291    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036293    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036296    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036299    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036301    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036304    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036306    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036309    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036311    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036314    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036316    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036319    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036321    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036324    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036326    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.037673 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036328    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036331    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036333    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036336    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036338    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036341    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036343    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036345    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036348    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036350    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036352    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036355    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036357    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036359    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036362    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036364    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036367    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036371    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036375    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.036391    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.038160 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037133    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037141    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037144    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037147    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037150    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037153    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037156    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037159    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037162    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037164    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037167    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037170    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037172    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037175    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037177    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037180    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037183    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037185    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037188    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037192    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.038643 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037196    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037199    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037201    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037204    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037206    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037210    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037215    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037221    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037225    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037229    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037233    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037238    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037243    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037246    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037248    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037251    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037253    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037256    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.039130 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037258    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037260    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037263    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037266    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037268    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037270    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037273    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037275    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037278    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037280    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037283    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037285    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037288    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037290    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037293    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037295    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037298    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037300    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037302    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037305    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.039571 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037308    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037310    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037312    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037315    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037318    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037322    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037325    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037327    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037330    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037332    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037335    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037337    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037339    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037342    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037344    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037347    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037349    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037351    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037354    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037356    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.040068 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037359    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037361    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037363    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037366    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037368    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037370    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037373    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037375    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037393    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037395    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037398    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037400    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037403    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037405    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037408    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037410    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037413    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037415    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037418    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037422    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.040563 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.037424    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038402    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038410    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038414    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038417    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038420    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038423    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038426    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038428    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038431    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038434    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038436    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038439    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038441    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038444    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038447    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038449    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038452    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038454    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038458    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.041033 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038462    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038465    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038469    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038472    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038475    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038478    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038481    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038484    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038488    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038491    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038494    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038498    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038501    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038503    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038506    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038508    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038510    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038513    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038515    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.041509 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038518    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038521    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038523    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038526    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038529    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038531    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038533    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038536    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038538    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038541    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038543    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038546    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038548    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038551    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038553    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038556    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038558    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038561    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038563    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.041985 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038566    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038568    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038571    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038573    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038576    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038579    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038582    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038584    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038586    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038590    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038593    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038595    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038598    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038600    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038603    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038605    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038607    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038610    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038612    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038614    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.042452 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038617    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038619    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038621    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038624    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038626    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038629    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038631    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038634    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038636    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038639    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038644    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038647    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038649    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038652    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038655    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038659    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038661    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038663    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038666    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038669    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.042934 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038671    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.038673    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038746    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038753    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038759    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038763    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038767    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038770    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038775    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038779    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038782    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038785    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038789    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038792    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038795    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038797    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038800    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038803    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038806    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038809    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038813    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038816    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038821    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038824    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 20:47:30.043403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038827    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038831    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038834    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038839    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038842    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038845    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038848    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038851    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038868    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038871    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038878    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038881    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038884    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038886    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038889    2581 flags.go:64] FLAG: --enable-server="true"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038892    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038896    2581 flags.go:64] FLAG: --event-burst="100"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038899    2581 flags.go:64] FLAG: --event-qps="50"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038901    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038904    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038907    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038911    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038913    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038916    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038919    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 29 20:47:30.043948 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038922    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038925    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038927    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038930    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038933    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038936    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038940    2581 flags.go:64] FLAG: --feature-gates=""
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038943    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038946    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038948    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038953    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038956    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038959    2581 flags.go:64] FLAG: --help="false"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038962    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038965    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038968    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038971    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038974    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038978    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038981    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038983    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038986    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038988    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 20:47:30.044544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038992    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038995    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.038997    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039000    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039003    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039006    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039008    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039011    2581 flags.go:64] FLAG: --lock-file=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039013    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039016    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039019    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039024    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039027    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039029    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039033    2581 flags.go:64] FLAG: --logging-format="text"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039036    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039039    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039042    2581 flags.go:64] FLAG: --manifest-url=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039045    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039050    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039053    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039057    2581 flags.go:64] FLAG: --max-pods="110"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039060    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039063    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039066    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 20:47:30.045265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039069    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039072    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039075    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039078    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039085    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039087    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039090    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039093    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039096    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039101    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039104    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039107    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039110    2581 flags.go:64] FLAG: --port="10250"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039113    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039116    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00841ac07178ec042"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039119    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039122    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039124    2581 flags.go:64] FLAG: --register-node="true"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039127    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039130    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039133    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039136    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039141    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039144    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039147    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 20:47:30.046297 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039150    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039153    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039156    2581 flags.go:64] FLAG: --runonce="false"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039159    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039161    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039164    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039167    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039170    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039188    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039192    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039195    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039198    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039201    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039204    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039206    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039209    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039212    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039214    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039220    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039222    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039225    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039231    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039234    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039236    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039239    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 20:47:30.047172 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039242    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039244    2581 flags.go:64] FLAG: --v="2"
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039248    2581 flags.go:64] FLAG: --version="false"
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039252    2581 flags.go:64] FLAG: --vmodule=""
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039256    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039259    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039371    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039375    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039391    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039395    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039398    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039400    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039403    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039406    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039408    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039411    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039414    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039417    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039420    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039423    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039426    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.047771 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039429    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039431    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039434    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039437    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039439    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039442    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039445    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039447    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039449    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039452    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039454    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039457    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039462    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039465    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039468    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039472    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039475    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039477    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039480    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039483    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.048262 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039486    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039488    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039492    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039495    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039498    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039501    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039504    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039506    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039509    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039512    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039514    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039517    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039519    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039522    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039524    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039526    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039529    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039531    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039534    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039536    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.048742 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039538    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039541    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039543    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039546    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039548    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039551    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039553    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039557    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039560    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039562    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039565    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039567    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039570    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039572    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039575    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039578    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039581    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039583    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039586    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039588    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.049220 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039591    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039593    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039596    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039599    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039601    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039604    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039606    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039608    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039611    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039613    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039615    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039618    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039620    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039622    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039625    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039627    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039629    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039632    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039634    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.049870 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039638    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039641    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039643    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039647    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.039652    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.039658    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.047822    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.047837    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.048793    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.048801    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048858    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048862    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048866    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048869    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.050318 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048872    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048874    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048877    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048879    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048882    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048884    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048887    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048890    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048892    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048894    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048897    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048899    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048902    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048905    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048907    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048910    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048912    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048915    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048917    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048920    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.050688 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048922    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048924    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048927    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048930    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048933    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048935    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048938    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048940    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048943    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048946    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048949    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048953    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048955    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048958    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048960    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048963    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048965    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048967    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048970    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.051170 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048972    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048975    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048977    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048980    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048985    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048988    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048991    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048994    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048997    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.048999    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049002    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049004    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049007    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049009    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049012    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049014    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049016    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049019    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049022    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.051648 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049024    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049027    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049030    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049032    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049034    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049037    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049039    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049042    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049044    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049047    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049049    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049052    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049054    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049057    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049059    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049063    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049066    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049069    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049072    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.052085 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049075    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049078    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049081    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049083    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049086    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049088    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049090    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049093    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049095    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049098    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049100    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049103    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049105    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049107    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049110    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049112    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049115    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.052665 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049117    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.049122    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049230    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049235    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049238    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049241    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049244    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049247    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049250    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049252    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049255    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049257    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049260    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049263    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049266    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:47:30.053088 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049269    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049271    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049274    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049277    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049279    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049282    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049284    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049286    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049288    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049291    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049293    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049296    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049298    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049301    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049303    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049306    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049309    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049312    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049315    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049317    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:47:30.053453 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049321    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049324    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049327    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049329    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049332    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049335    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049337    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049340    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049342    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049345    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049347    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049350    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049352    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049355    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049358    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049360    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049362    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049365    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049367    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049370    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:47:30.053945 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049372    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049374    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049377    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049391    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049394    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049396    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049399    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049401    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049403    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049406    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049408    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049411    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049413    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049415    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049418    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049420    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049422    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049425    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049427    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:47:30.054449 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049430    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049432    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049435    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049437    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049440    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049442    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049448    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049451    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049453    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049455    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049458    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049460    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049463    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049465    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049468    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049470    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049473    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049475    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049479    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:47:30.054919 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049482    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049484    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049487    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049490    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049492    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049495    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049498    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:30.049500    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.049505    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.049626    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.051547    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.052779    2581 server.go:1019] "Starting client certificate rotation"
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.052881    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 20:47:30.055350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.052907    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 20:47:30.055672 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.055199    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 20:47:30.056700 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.056685    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 20:47:30.069706 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.069690    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 29 20:47:30.072041 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.072025    2581 log.go:25] "Validated CRI v1 image API"
Jul 29 20:47:30.073371 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.073355    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 20:47:30.076557 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.076537    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c10dfd9f-857a-4728-a6ea-418e80e4475f:/dev/nvme0n1p3 cc24ca87-7c7e-49c2-9b9c-25eb0a09e703:/dev/nvme0n1p4]
Jul 29 20:47:30.076623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.076556    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 20:47:30.081458 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.081042    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 20:47:30.083158 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.083051    2581 manager.go:216] Machine: {Timestamp:2026-07-29 20:47:30.081798833 +0000 UTC m=+0.331036340 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200418 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261d49b0a4f2ea73c5011e0d5f8097 SystemUUID:ec261d49-b0a4-f2ea-73c5-011e0d5f8097 BootID:9b1cd9d8-1bb2-4fda-bbca-fa7f2d61c56f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:59:27:c7:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:59:27:c7:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:77:33:38:c1:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 20:47:30.083158 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.083149    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 20:47:30.083327 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.083255    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 20:47:30.086951 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.086921    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 20:47:30.087098 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.086954    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 20:47:30.087618 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.087606    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 20:47:30.087682 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.087624    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 20:47:30.087682 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.087634    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 20:47:30.088915 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.088902    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 20:47:30.089647 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.089636    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 20:47:30.091539 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091525    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 20:47:30.091676 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091666    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 20:47:30.091739 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091703    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 20:47:30.091739 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091718    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 20:47:30.091739 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091733    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 20:47:30.091859 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091746    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 29 20:47:30.091859 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.091758    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 20:47:30.092151 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.092126    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 20:47:30.092151 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.092151    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 20:47:30.095643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.095627    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 20:47:30.097180 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.097164    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 20:47:30.097242 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.097198    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 20:47:30.098890 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098868    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 20:47:30.098890 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098891    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098900    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098909    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098920    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098929    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098940    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098949    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098956    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098962    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098980    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 20:47:30.099009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.098986    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 20:47:30.099813 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.099802    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 20:47:30.099813 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.099814    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 20:47:30.103054 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.103016    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 20:47:30.103054 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.103037    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 20:47:30.103173 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.103095    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 20:47:30.103173 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.103114    2581 server.go:1268] "Started kubelet"
Jul 29 20:47:30.103263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.103182    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 20:47:30.103263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.103178    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 20:47:30.103263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.103251    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 20:47:30.103991 ip-10-0-135-229 systemd[1]: Started Kubernetes Kubelet.
Jul 29 20:47:30.104934 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.104909    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 20:47:30.105677 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.105662    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 20:47:30.109787 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.109768    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 20:47:30.114426 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.114406    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 20:47:30.114520 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.114433    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 20:47:30.114576 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.114520    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.115012 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.114996    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 20:47:30.115012 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115010    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 20:47:30.115143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115013    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 20:47:30.115143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115119    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 20:47:30.115143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115128    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 20:47:30.115417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115402    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 20:47:30.115417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115417    2581 factory.go:55] Registering systemd factory
Jul 29 20:47:30.115516 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115423    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 29 20:47:30.115698 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115681    2581 factory.go:153] Registering CRI-O factory
Jul 29 20:47:30.115698 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115700    2581 factory.go:223] Registration of the crio container factory successfully
Jul 29 20:47:30.115817 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115734    2581 factory.go:103] Registering Raw factory
Jul 29 20:47:30.115817 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.115748    2581 manager.go:1196] Started watching for new ooms in manager
Jul 29 20:47:30.116171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.116155    2581 manager.go:349] Starting recovery of all containers
Jul 29 20:47:30.117437 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.116423    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-229.ec2.internal.18c6ddaff543a1e2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-229.ec2.internal,UID:ip-10-0-135-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-229.ec2.internal,},FirstTimestamp:2026-07-29 20:47:30.102845922 +0000 UTC m=+0.352083485,LastTimestamp:2026-07-29 20:47:30.102845922 +0000 UTC m=+0.352083485,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-229.ec2.internal,}"
Jul 29 20:47:30.127492 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.127472    2581 manager.go:354] Recovery completed
Jul 29 20:47:30.127735 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.127704    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 20:47:30.127817 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.127757    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 20:47:30.131929 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.131916    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.133719 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.133688    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.133811 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.133734    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.133811 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.133763    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.134288 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.134265    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 20:47:30.134288 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.134280    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.134427 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.134285    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 20:47:30.134427 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.134349    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 20:47:30.135003 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.134986    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.135078 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.135018    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.135078 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.135033    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.135979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.135965    2581 policy_none.go:49] "None policy: Start"
Jul 29 20:47:30.135979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.135981    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 20:47:30.136063 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.135991    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 20:47:30.137123 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.137113    2581 policy_none.go:47] "Start"
Jul 29 20:47:30.139590 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.139575    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lkpzh"
Jul 29 20:47:30.144613 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.144554    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-229.ec2.internal.18c6ddaff71aba68  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-229.ec2.internal,UID:ip-10-0-135-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-229.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-229.ec2.internal,},FirstTimestamp:2026-07-29 20:47:30.133719656 +0000 UTC m=+0.382957148,LastTimestamp:2026-07-29 20:47:30.133719656 +0000 UTC m=+0.382957148,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-229.ec2.internal,}"
Jul 29 20:47:30.148309 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.148293    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lkpzh"
Jul 29 20:47:30.152768 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.152707    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-229.ec2.internal.18c6ddaff71b073c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-229.ec2.internal,UID:ip-10-0-135-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-229.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-229.ec2.internal,},FirstTimestamp:2026-07-29 20:47:30.133739324 +0000 UTC m=+0.382976829,LastTimestamp:2026-07-29 20:47:30.133739324 +0000 UTC m=+0.382976829,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-229.ec2.internal,}"
Jul 29 20:47:30.175004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.174989    2581 manager.go:337] "Starting Device Plugin manager"
Jul 29 20:47:30.175073 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.175019    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 20:47:30.175073 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175029    2581 server.go:87] "Starting device plugin registration server"
Jul 29 20:47:30.175342 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175329    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 20:47:30.175417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175343    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 20:47:30.175459 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175446    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 20:47:30.175535 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175524    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 20:47:30.175567 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.175538    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 20:47:30.175997 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.175975    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 20:47:30.176096 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.176026    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.255017 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.254971    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 20:47:30.256488 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.256469    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 20:47:30.256582 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.256509    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 20:47:30.256582 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.256530    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 20:47:30.256582 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.256564    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 20:47:30.259479 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.259454    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:47:30.275440 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.275420    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.276312 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.276292    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.276402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.276328    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.276402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.276343    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.276402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.276364    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.286722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.286653    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.286722 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.286676    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-229.ec2.internal\": node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.317662 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.317636    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.356873 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.356836    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"]
Jul 29 20:47:30.357010 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.356925    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.358505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.358488    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.358592 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.358517    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.358592 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.358535    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.359704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.359692    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.359814 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.359799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.359852 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.359831    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.360425 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360410    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.360494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360410    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.360494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360465    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.360494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360478    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.360494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360440    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.360615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.360509    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.361932 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.361915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.362008 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.361951    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:30.362602 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.362588    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:30.362683 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.362614    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:30.362683 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.362628    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:30.387078 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.387055    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-229.ec2.internal\" not found" node="ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.391573 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.391559    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-229.ec2.internal\" not found" node="ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.417122 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.417100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c882f98f11e192b3a9e8390a2cecc279-config\") pod \"kube-apiserver-proxy-ip-10-0-135-229.ec2.internal\" (UID: \"c882f98f11e192b3a9e8390a2cecc279\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.417208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.417124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.417208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.417143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.418141 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.418126    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.517417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.517417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.517615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.517615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3dea8349b1cf67101da549d62710210-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal\" (UID: \"b3dea8349b1cf67101da549d62710210\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.517615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c882f98f11e192b3a9e8390a2cecc279-config\") pod \"kube-apiserver-proxy-ip-10-0-135-229.ec2.internal\" (UID: \"c882f98f11e192b3a9e8390a2cecc279\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.517615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.517516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c882f98f11e192b3a9e8390a2cecc279-config\") pod \"kube-apiserver-proxy-ip-10-0-135-229.ec2.internal\" (UID: \"c882f98f11e192b3a9e8390a2cecc279\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.518438 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.518421    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.619259 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.619187    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.689347 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.689322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.695494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:30.695343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:30.719444 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.719412    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.819891 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.819847    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:30.920343 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:30.920267    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:31.020771 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:31.020725    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:31.052901 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.052876    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 20:47:31.053291 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.053037    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 20:47:31.120857 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:31.120817    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:31.152021 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.151990    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 20:42:30 +0000 UTC" deadline="2028-04-15 15:07:09.86690322 +0000 UTC"
Jul 29 20:47:31.152021 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.152019    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15018h19m38.714887357s"
Jul 29 20:47:31.221605 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:31.221546    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:31.229635 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.229618    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:47:31.231005 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.230988    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:47:31.231106 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.231023    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:47:31.231106 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.231037    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:47:31.322104 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:31.322076    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-229.ec2.internal\" not found"
Jul 29 20:47:31.322669 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:31.322629    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc882f98f11e192b3a9e8390a2cecc279.slice/crio-9be44cb4620d45e2bc17c4f37e6085470656e90bbd3cf10b53d3ce2a8261eaa2 WatchSource:0}: Error finding container 9be44cb4620d45e2bc17c4f37e6085470656e90bbd3cf10b53d3ce2a8261eaa2: Status 404 returned error can't find the container with id 9be44cb4620d45e2bc17c4f37e6085470656e90bbd3cf10b53d3ce2a8261eaa2
Jul 29 20:47:31.323008 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:31.322992    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3dea8349b1cf67101da549d62710210.slice/crio-cfe6cdc6fa2cb9b3ee5a00077f188c7540711652f9a8e9d2bf646981cf68633b WatchSource:0}: Error finding container cfe6cdc6fa2cb9b3ee5a00077f188c7540711652f9a8e9d2bf646981cf68633b: Status 404 returned error can't find the container with id cfe6cdc6fa2cb9b3ee5a00077f188c7540711652f9a8e9d2bf646981cf68633b
Jul 29 20:47:31.328986 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.328963    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:47:31.374012 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.373982    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:47:31.414688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.414658    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:31.415428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.415411    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:47:31.428358 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.428340    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 20:47:31.429647 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.429635    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal"
Jul 29 20:47:31.436344 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.436329    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 20:47:31.609051 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:31.608961    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:47:32.092366 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.092257    2581 apiserver.go:52] "Watching apiserver"
Jul 29 20:47:32.099585 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.099550    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 20:47:32.100187 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.100148    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-j7vmc","openshift-cluster-node-tuning-operator/tuned-k2qm7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal","openshift-multus/multus-v9b6d","openshift-multus/network-metrics-daemon-ldb4s","openshift-network-operator/iptables-alerter-s8xpd","openshift-ovn-kubernetes/ovnkube-node-bm4tw","kube-system/global-pull-secret-syncer-h78gc","kube-system/konnectivity-agent-68k2p","kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2","openshift-dns/node-resolver-dl4bj","openshift-image-registry/node-ca-zwpjs","openshift-multus/multus-additional-cni-plugins-nxnjd"]
Jul 29 20:47:32.103142 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.103117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:32.103254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.103205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:32.105314 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.105292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.109971 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.109948    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 20:47:32.112899 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.112873    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-76l8q\""
Jul 29 20:47:32.113004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.112940    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 20:47:32.113969 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.113947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.114889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.114852    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 20:47:32.116629 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.116601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.117842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.117819    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 20:47:32.117955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.117933    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 20:47:32.118073 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.118052    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.118176 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.118156    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kq7lr\""
Jul 29 20:47:32.118250 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.118217    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 20:47:32.118426 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.118407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.118627 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.118610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 20:47:32.119516 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.119497    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.120292 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.120273    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.120364 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.120321    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 20:47:32.120695 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.120676    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-trjrn\""
Jul 29 20:47:32.120769 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.120676    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 20:47:32.121879 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.121858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.121993 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.121965    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:32.126123 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.126103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.126606 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.126585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-kubelet\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.126749 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.126730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.126915 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.126897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-bin\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-system-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127182 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-k8s-cni-cncf-io\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127182 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-multus\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127182 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-systemd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127328 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-ovn\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127328 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-node-log\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127512 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-config\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127512 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cnibin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127512 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-kubelet\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127512 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-etc-kubernetes\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-systemd-units\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-script-lib\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-socket-dir-parent\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-multus-certs\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.127903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:32.127903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97471526-67b0-415f-a32a-fdcda1de72ce-konnectivity-ca\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.127903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-netns\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.127903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwjxb\" (UniqueName: \"kubernetes.io/projected/c18336fb-f2d2-4f88-99af-124a2f4f1539-kube-api-access-wwjxb\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-os-release\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.127957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-hostroot\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-var-lib-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128294 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-log-socket\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128347 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-env-overrides\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128347 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9vql\" (UniqueName: \"kubernetes.io/projected/4199ec18-e565-4629-bcab-ad9fffdc2aa9-kube-api-access-f9vql\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovn-node-metrics-cert\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-bin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-daemon-config\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97471526-67b0-415f-a32a-fdcda1de72ce-agent-certs\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-slash\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-etc-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-netd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cni-binary-copy\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.128668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-netns\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.128971 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.128674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-conf-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.130903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.130879    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.131049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.130879    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.131134 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.131119    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hbw52\""
Jul 29 20:47:32.131359 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.131324    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 20:47:32.131473 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.131442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.131562 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.131539    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:32.134945 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.134904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.140875 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.140847    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.141146 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.141127    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.141224 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.141147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.141281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.141263    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pq97p\""
Jul 29 20:47:32.142366 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.142347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.143901 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.143833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tdm7v\""
Jul 29 20:47:32.144010 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.143991    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 20:47:32.144100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.144082    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.144256 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.144207    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.144583 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.144564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p28r9\""
Jul 29 20:47:32.146238 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.144615    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.146238 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.145552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.146238 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.145921    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.148552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.148111    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 20:47:32.148552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.148174    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2ljhc\""
Jul 29 20:47:32.148552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.148449    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 20:47:32.148552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.148456    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 20:47:32.148813 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.148794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.150920 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.150899    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 20:47:32.151448 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.151179    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zvfsr\""
Jul 29 20:47:32.151448 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.151188    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 20:47:32.202306 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.202277    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 20:47:32.218649 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.218618    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xb898"
Jul 29 20:47:32.218858 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.218842    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 20:47:32.229640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-kubelet\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.229797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.229797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-bin\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.229797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-system-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.229797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-multus\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.229797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-run\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-lib-modules\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-tuned\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-config\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-etc-kubernetes\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-os-release\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca19729a-23ec-47a7-8cc5-279953c8869f-host-slash\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-kubelet-config\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.229999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfv79\" (UniqueName: \"kubernetes.io/projected/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-kube-api-access-jfv79\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-registration-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-multus-certs\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-system-cni-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-sys\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97471526-67b0-415f-a32a-fdcda1de72ce-konnectivity-ca\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94gdf\" (UniqueName: \"kubernetes.io/projected/1769ec3c-462f-4fb0-86f9-fcecf9756677-kube-api-access-94gdf\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-modprobe-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml4mx\" (UniqueName: \"kubernetes.io/projected/c1220861-03f4-47ec-9fa7-56b93705461e-kube-api-access-ml4mx\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-var-lib-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cbea866-730e-4c2c-b0a2-e16bfb489af8-serviceca\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.230508 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztslc\" (UniqueName: \"kubernetes.io/projected/8cbea866-730e-4c2c-b0a2-e16bfb489af8-kube-api-access-ztslc\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-sys-fs\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-slash\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-netd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cni-binary-copy\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x29f2\" (UniqueName: \"kubernetes.io/projected/ca19729a-23ec-47a7-8cc5-279953c8869f-kube-api-access-x29f2\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-kubernetes\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-multus-certs\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdnq\" (UniqueName: \"kubernetes.io/projected/8b7f12a9-0f65-492a-8764-a3d0ace547fe-kube-api-access-2hdnq\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-var-lib-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-bin\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-slash\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-cni-netd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.230975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-multus\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.231228 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-k8s-cni-cncf-io\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-device-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231099    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xb898"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-systemd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-ovn\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-node-log\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cnibin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-kubelet\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-conf\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9327713-5426-4824-88bd-2b849ff5997f-tmp-dir\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-systemd-units\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-script-lib\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-socket-dir-parent\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-netns\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwjxb\" (UniqueName: \"kubernetes.io/projected/c18336fb-f2d2-4f88-99af-124a2f4f1539-kube-api-access-wwjxb\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-os-release\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-hostroot\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca19729a-23ec-47a7-8cc5-279953c8869f-iptables-alerter-script\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-dbus\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-log-socket\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-env-overrides\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9vql\" (UniqueName: \"kubernetes.io/projected/4199ec18-e565-4629-bcab-ad9fffdc2aa9-kube-api-access-f9vql\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovn-node-metrics-cert\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cni-binary-copy\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-config\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231806    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-etc-kubernetes\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-bin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-daemon-config\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-system-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cbea866-730e-4c2c-b0a2-e16bfb489af8-host\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-tmp\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.232850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9327713-5426-4824-88bd-2b849ff5997f-hosts-file\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.231984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mdv5\" (UniqueName: \"kubernetes.io/projected/f9327713-5426-4824-88bd-2b849ff5997f-kube-api-access-6mdv5\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97471526-67b0-415f-a32a-fdcda1de72ce-agent-certs\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-etc-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-cni-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-cnibin\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysconfig\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-var-lib-kubelet\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-socket-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-netns\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovnkube-script-lib\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-hostroot\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-conf-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-socket-dir-parent\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-run-netns\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-os-release\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-conf-dir\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.233903 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232669    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97471526-67b0-415f-a32a-fdcda1de72ce-konnectivity-ca\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-cnibin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-ovn\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-node-log\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-run-systemd\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-systemd-units\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-kubelet\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.232999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-netns\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-run-k8s-cni-cncf-io\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4199ec18-e565-4629-bcab-ad9fffdc2aa9-host-var-lib-cni-bin\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-host-kubelet\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233134    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c18336fb-f2d2-4f88-99af-124a2f4f1539-env-overrides\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-log-socket\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c18336fb-f2d2-4f88-99af-124a2f4f1539-etc-openvswitch\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-systemd\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.234799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4199ec18-e565-4629-bcab-ad9fffdc2aa9-multus-daemon-config\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.235626 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-host\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.235626 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.233362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.237880 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.237858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c18336fb-f2d2-4f88-99af-124a2f4f1539-ovn-node-metrics-cert\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.238801 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.238775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97471526-67b0-415f-a32a-fdcda1de72ce-agent-certs\") pod \"konnectivity-agent-68k2p\" (UID: \"97471526-67b0-415f-a32a-fdcda1de72ce\") " pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.245854 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.245833    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:32.245959 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.245860    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:32.245959 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.245874    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:32.245959 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.245947    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:47:32.74592529 +0000 UTC m=+2.995162808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:32.248388 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.248346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwjxb\" (UniqueName: \"kubernetes.io/projected/c18336fb-f2d2-4f88-99af-124a2f4f1539-kube-api-access-wwjxb\") pod \"ovnkube-node-bm4tw\" (UID: \"c18336fb-f2d2-4f88-99af-124a2f4f1539\") " pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.249175 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.249152    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9vql\" (UniqueName: \"kubernetes.io/projected/4199ec18-e565-4629-bcab-ad9fffdc2aa9-kube-api-access-f9vql\") pod \"multus-v9b6d\" (UID: \"4199ec18-e565-4629-bcab-ad9fffdc2aa9\") " pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.269951 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.269886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal" event={"ID":"c882f98f11e192b3a9e8390a2cecc279","Type":"ContainerStarted","Data":"9be44cb4620d45e2bc17c4f37e6085470656e90bbd3cf10b53d3ce2a8261eaa2"}
Jul 29 20:47:32.271961 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.271932    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal" event={"ID":"b3dea8349b1cf67101da549d62710210","Type":"ContainerStarted","Data":"cfe6cdc6fa2cb9b3ee5a00077f188c7540711652f9a8e9d2bf646981cf68633b"}
Jul 29 20:47:32.333721 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cbea866-730e-4c2c-b0a2-e16bfb489af8-host\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.333721 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-tmp\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9327713-5426-4824-88bd-2b849ff5997f-hosts-file\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mdv5\" (UniqueName: \"kubernetes.io/projected/f9327713-5426-4824-88bd-2b849ff5997f-kube-api-access-6mdv5\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-cnibin\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysconfig\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-var-lib-kubelet\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-socket-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-systemd\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.333980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-host\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-run\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-lib-modules\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-tuned\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-os-release\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca19729a-23ec-47a7-8cc5-279953c8869f-host-slash\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-kubelet-config\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfv79\" (UniqueName: \"kubernetes.io/projected/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-kube-api-access-jfv79\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-registration-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-system-cni-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-sys\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94gdf\" (UniqueName: \"kubernetes.io/projected/1769ec3c-462f-4fb0-86f9-fcecf9756677-kube-api-access-94gdf\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-modprobe-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml4mx\" (UniqueName: \"kubernetes.io/projected/c1220861-03f4-47ec-9fa7-56b93705461e-kube-api-access-ml4mx\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.334533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cbea866-730e-4c2c-b0a2-e16bfb489af8-serviceca\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztslc\" (UniqueName: \"kubernetes.io/projected/8cbea866-730e-4c2c-b0a2-e16bfb489af8-kube-api-access-ztslc\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-sys-fs\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x29f2\" (UniqueName: \"kubernetes.io/projected/ca19729a-23ec-47a7-8cc5-279953c8869f-kube-api-access-x29f2\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-kubernetes\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdnq\" (UniqueName: \"kubernetes.io/projected/8b7f12a9-0f65-492a-8764-a3d0ace547fe-kube-api-access-2hdnq\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-device-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-conf\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9327713-5426-4824-88bd-2b849ff5997f-tmp-dir\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca19729a-23ec-47a7-8cc5-279953c8869f-iptables-alerter-script\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-dbus\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.334963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.335149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cbea866-730e-4c2c-b0a2-e16bfb489af8-host\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335433    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-system-cni-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-sys-fs\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9327713-5426-4824-88bd-2b849ff5997f-hosts-file\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335580    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-sys\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-modprobe-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.335818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-cnibin\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.336110 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.335940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysconfig\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.336110 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-var-lib-kubelet\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.336110 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.336263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.336263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-socket-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.336263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-os-release\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.336405 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8cbea866-730e-4c2c-b0a2-e16bfb489af8-serviceca\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.336405 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca19729a-23ec-47a7-8cc5-279953c8869f-host-slash\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.336532 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336416    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-d\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.336532 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.336522    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:32.336623 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.336595    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:47:32.83657437 +0000 UTC m=+3.085811880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:32.336692 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1769ec3c-462f-4fb0-86f9-fcecf9756677-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-device-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-kubelet-config\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1769ec3c-462f-4fb0-86f9-fcecf9756677-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.336909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-registration-dir\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.336940    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.337025    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:47:32.837004904 +0000 UTC m=+3.086242404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-kubernetes\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-sysctl-conf\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-systemd\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9327713-5426-4824-88bd-2b849ff5997f-tmp-dir\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-run\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1220861-03f4-47ec-9fa7-56b93705461e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45771b-8495-4116-981b-9dcf8cffedcd-dbus\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-host\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337330    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-lib-modules\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.337768 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.337702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca19729a-23ec-47a7-8cc5-279953c8869f-iptables-alerter-script\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.338569 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.338082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-tmp\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.338623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.338572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-etc-tuned\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.355793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.355683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml4mx\" (UniqueName: \"kubernetes.io/projected/c1220861-03f4-47ec-9fa7-56b93705461e-kube-api-access-ml4mx\") pod \"aws-ebs-csi-driver-node-2wkh2\" (UID: \"c1220861-03f4-47ec-9fa7-56b93705461e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.359800 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.359559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfv79\" (UniqueName: \"kubernetes.io/projected/9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f-kube-api-access-jfv79\") pod \"tuned-k2qm7\" (UID: \"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f\") " pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.359800 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.359663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdnq\" (UniqueName: \"kubernetes.io/projected/8b7f12a9-0f65-492a-8764-a3d0ace547fe-kube-api-access-2hdnq\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.360030 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.359995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztslc\" (UniqueName: \"kubernetes.io/projected/8cbea866-730e-4c2c-b0a2-e16bfb489af8-kube-api-access-ztslc\") pod \"node-ca-zwpjs\" (UID: \"8cbea866-730e-4c2c-b0a2-e16bfb489af8\") " pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.360559 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.360532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94gdf\" (UniqueName: \"kubernetes.io/projected/1769ec3c-462f-4fb0-86f9-fcecf9756677-kube-api-access-94gdf\") pod \"multus-additional-cni-plugins-nxnjd\" (UID: \"1769ec3c-462f-4fb0-86f9-fcecf9756677\") " pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.360682 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.360589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mdv5\" (UniqueName: \"kubernetes.io/projected/f9327713-5426-4824-88bd-2b849ff5997f-kube-api-access-6mdv5\") pod \"node-resolver-dl4bj\" (UID: \"f9327713-5426-4824-88bd-2b849ff5997f\") " pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.361778 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.361751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x29f2\" (UniqueName: \"kubernetes.io/projected/ca19729a-23ec-47a7-8cc5-279953c8869f-kube-api-access-x29f2\") pod \"iptables-alerter-s8xpd\" (UID: \"ca19729a-23ec-47a7-8cc5-279953c8869f\") " pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.408793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.408755    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:47:32.426037 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.425981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:32.433086 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.433056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:32.439926 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.439900    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97471526_67b0_415f_a32a_fdcda1de72ce.slice/crio-5dd3b8f48cd87d675f2a2f0316a364c3a090957ec3290016773af444ce3bb52b WatchSource:0}: Error finding container 5dd3b8f48cd87d675f2a2f0316a364c3a090957ec3290016773af444ce3bb52b: Status 404 returned error can't find the container with id 5dd3b8f48cd87d675f2a2f0316a364c3a090957ec3290016773af444ce3bb52b
Jul 29 20:47:32.447323 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.447293    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18336fb_f2d2_4f88_99af_124a2f4f1539.slice/crio-bd4ca024612f377056d7ecf64879376f53d6125cab0f783bf07d8db5b28b9301 WatchSource:0}: Error finding container bd4ca024612f377056d7ecf64879376f53d6125cab0f783bf07d8db5b28b9301: Status 404 returned error can't find the container with id bd4ca024612f377056d7ecf64879376f53d6125cab0f783bf07d8db5b28b9301
Jul 29 20:47:32.453120 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.453092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v9b6d"
Jul 29 20:47:32.459865 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.459431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8xpd"
Jul 29 20:47:32.465242 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.465211    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4199ec18_e565_4629_bcab_ad9fffdc2aa9.slice/crio-3578e1933789dbbea54aa0d1e699291c2157880f81c4d3a3d48742197e216d2b WatchSource:0}: Error finding container 3578e1933789dbbea54aa0d1e699291c2157880f81c4d3a3d48742197e216d2b: Status 404 returned error can't find the container with id 3578e1933789dbbea54aa0d1e699291c2157880f81c4d3a3d48742197e216d2b
Jul 29 20:47:32.468378 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.468354    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k2qm7"
Jul 29 20:47:32.471710 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.471674    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca19729a_23ec_47a7_8cc5_279953c8869f.slice/crio-9cf0bbe20aea7cc4669b44530b40021dbb24201b30ee6c37f6b3d44de28c2327 WatchSource:0}: Error finding container 9cf0bbe20aea7cc4669b44530b40021dbb24201b30ee6c37f6b3d44de28c2327: Status 404 returned error can't find the container with id 9cf0bbe20aea7cc4669b44530b40021dbb24201b30ee6c37f6b3d44de28c2327
Jul 29 20:47:32.477408 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.477337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2"
Jul 29 20:47:32.481062 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.481035    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2bd3e8_d894_4e5a_b11f_c4dc39666d7f.slice/crio-a7f21d52bea57988ff5803c523c539b167a1cb0cb765c41df1dc0c22daff6dd7 WatchSource:0}: Error finding container a7f21d52bea57988ff5803c523c539b167a1cb0cb765c41df1dc0c22daff6dd7: Status 404 returned error can't find the container with id a7f21d52bea57988ff5803c523c539b167a1cb0cb765c41df1dc0c22daff6dd7
Jul 29 20:47:32.486308 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.486291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dl4bj"
Jul 29 20:47:32.494660 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.494633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zwpjs"
Jul 29 20:47:32.496375 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.496347    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1220861_03f4_47ec_9fa7_56b93705461e.slice/crio-e169da23d8652624806177115da8d59c1f44d6b627c8ab64260feb9a8efd4fc6 WatchSource:0}: Error finding container e169da23d8652624806177115da8d59c1f44d6b627c8ab64260feb9a8efd4fc6: Status 404 returned error can't find the container with id e169da23d8652624806177115da8d59c1f44d6b627c8ab64260feb9a8efd4fc6
Jul 29 20:47:32.502507 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.502469    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9327713_5426_4824_88bd_2b849ff5997f.slice/crio-a980a433858aa4cde8da7a7b572e1bd6b84f70bffe038e736e8d00649ba18615 WatchSource:0}: Error finding container a980a433858aa4cde8da7a7b572e1bd6b84f70bffe038e736e8d00649ba18615: Status 404 returned error can't find the container with id a980a433858aa4cde8da7a7b572e1bd6b84f70bffe038e736e8d00649ba18615
Jul 29 20:47:32.502610 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.502573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxnjd"
Jul 29 20:47:32.509392 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.509361    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbea866_730e_4c2c_b0a2_e16bfb489af8.slice/crio-1ac69d2f614b13080d0c0527a237392ac6f82375349e2eaf5e442ec1d12cc390 WatchSource:0}: Error finding container 1ac69d2f614b13080d0c0527a237392ac6f82375349e2eaf5e442ec1d12cc390: Status 404 returned error can't find the container with id 1ac69d2f614b13080d0c0527a237392ac6f82375349e2eaf5e442ec1d12cc390
Jul 29 20:47:32.516845 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:47:32.516819    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1769ec3c_462f_4fb0_86f9_fcecf9756677.slice/crio-56678d06569d49b223daf002a88d6aec1e18117f3d4dcd0adfe255e6b3b2ecc6 WatchSource:0}: Error finding container 56678d06569d49b223daf002a88d6aec1e18117f3d4dcd0adfe255e6b3b2ecc6: Status 404 returned error can't find the container with id 56678d06569d49b223daf002a88d6aec1e18117f3d4dcd0adfe255e6b3b2ecc6
Jul 29 20:47:32.839274 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.839177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:32.839274 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.839241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:32.839300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839377    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839436    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839487    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.839458316 +0000 UTC m=+4.088695825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839486    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839513    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839520    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.839501698 +0000 UTC m=+4.088739232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:32.839531 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839526    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:32.839885 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:32.839573    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.839558153 +0000 UTC m=+4.088795664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:33.237267 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.237063    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:42:32 +0000 UTC" deadline="2028-04-04 13:30:52.788616331 +0000 UTC"
Jul 29 20:47:33.237267 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.237153    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14752h43m19.551469365s"
Jul 29 20:47:33.256813 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.256768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:33.257078 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.256945    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:33.276964 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.276921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68k2p" event={"ID":"97471526-67b0-415f-a32a-fdcda1de72ce","Type":"ContainerStarted","Data":"5dd3b8f48cd87d675f2a2f0316a364c3a090957ec3290016773af444ce3bb52b"}
Jul 29 20:47:33.278416 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.278299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2" event={"ID":"c1220861-03f4-47ec-9fa7-56b93705461e","Type":"ContainerStarted","Data":"e169da23d8652624806177115da8d59c1f44d6b627c8ab64260feb9a8efd4fc6"}
Jul 29 20:47:33.284719 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.284643    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8xpd" event={"ID":"ca19729a-23ec-47a7-8cc5-279953c8869f","Type":"ContainerStarted","Data":"9cf0bbe20aea7cc4669b44530b40021dbb24201b30ee6c37f6b3d44de28c2327"}
Jul 29 20:47:33.286820 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.286751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerStarted","Data":"56678d06569d49b223daf002a88d6aec1e18117f3d4dcd0adfe255e6b3b2ecc6"}
Jul 29 20:47:33.288744 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.288698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zwpjs" event={"ID":"8cbea866-730e-4c2c-b0a2-e16bfb489af8","Type":"ContainerStarted","Data":"1ac69d2f614b13080d0c0527a237392ac6f82375349e2eaf5e442ec1d12cc390"}
Jul 29 20:47:33.290168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.290090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dl4bj" event={"ID":"f9327713-5426-4824-88bd-2b849ff5997f","Type":"ContainerStarted","Data":"a980a433858aa4cde8da7a7b572e1bd6b84f70bffe038e736e8d00649ba18615"}
Jul 29 20:47:33.291803 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.291745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k2qm7" event={"ID":"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f","Type":"ContainerStarted","Data":"a7f21d52bea57988ff5803c523c539b167a1cb0cb765c41df1dc0c22daff6dd7"}
Jul 29 20:47:33.293123 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.293073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v9b6d" event={"ID":"4199ec18-e565-4629-bcab-ad9fffdc2aa9","Type":"ContainerStarted","Data":"3578e1933789dbbea54aa0d1e699291c2157880f81c4d3a3d48742197e216d2b"}
Jul 29 20:47:33.295549 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.295379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"bd4ca024612f377056d7ecf64879376f53d6125cab0f783bf07d8db5b28b9301"}
Jul 29 20:47:33.849142 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.849044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:33.849142 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.849121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:33.849381 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:33.849160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:33.849381 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849369    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:33.849530 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849459    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:47:35.849421748 +0000 UTC m=+6.098659267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:33.849530 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849476    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:33.849530 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849495    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:33.849530 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849508    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:33.849756 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849551    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:47:35.849538645 +0000 UTC m=+6.098776140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:33.849756 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849616    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:33.849756 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:33.849649    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:47:35.849637552 +0000 UTC m=+6.098875064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:34.237542 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:34.237405    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:42:32 +0000 UTC" deadline="2028-04-06 00:17:49.625209626 +0000 UTC"
Jul 29 20:47:34.237542 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:34.237480    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14787h30m15.387733748s"
Jul 29 20:47:34.257745 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:34.257702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:34.257963 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:34.257876    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:34.258543 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:34.258513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:34.258649 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:34.258628    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:35.257378 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:35.257340    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:35.257921 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.257513    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:35.868021 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:35.867976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:35.868220 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:35.868055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:35.868220 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:35.868096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:35.868375 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868252    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:35.868375 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868319    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:47:39.868298954 +0000 UTC m=+10.117536465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:35.868891 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868871    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:35.868984 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868902    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:35.868984 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868916    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:35.868984 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.868964    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:47:39.868948265 +0000 UTC m=+10.118185763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:35.869166 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.869041    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:35.869166 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:35.869077    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:47:39.869063629 +0000 UTC m=+10.118301130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:36.260569 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:36.259678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:36.260569 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:36.259828    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:36.260569 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:36.260254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:36.260569 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:36.260365    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:37.257182 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:37.257144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:37.257362 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:37.257307    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:38.261600 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:38.261554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:38.261600 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:38.261595    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:38.262148 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:38.261771    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:38.262293 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:38.262252    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:39.257313 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:39.257269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:39.257549 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.257445    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:39.903361 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:39.903320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:39.903400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:39.903461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.903627    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.903650    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.903663    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:39.903859 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.903721    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:47:47.90370435 +0000 UTC m=+18.152941846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:39.904139 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.904107    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:39.904179 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.904155    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:47:47.904142779 +0000 UTC m=+18.153380271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:39.904266 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.904222    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:39.904266 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:39.904252    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:47:47.904244529 +0000 UTC m=+18.153482023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:40.258351 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:40.258274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:40.258549 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:40.258365    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:40.258850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:40.258825    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:40.258979 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:40.258930    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:41.257668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:41.257635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:41.258123 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:41.257772    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:42.258544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:42.258508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:42.258968 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:42.258654    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:42.259023 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:42.258508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:42.259146 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:42.259123    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:43.257553 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:43.257517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:43.257721 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:43.257621    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:44.257303 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:44.257223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:44.257657 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:44.257336    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:44.257657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:44.257404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:44.257657 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:44.257512    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:45.256944 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:45.256901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:45.257136 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:45.257045    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:46.257560 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:46.257522    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:46.257924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:46.257572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:46.257924 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:46.257647    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:46.257924 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:46.257774    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:47.257178 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:47.257141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:47.257458 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.257255    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:47.965857 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:47.965813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:47.966001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966010    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:47.966046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966091    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:48:03.96606959 +0000 UTC m=+34.215307105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966151    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966164    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966181    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966191    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966194    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:48:03.966183063 +0000 UTC m=+34.215420557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:47.966316 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:47.966222    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:48:03.966213318 +0000 UTC m=+34.215450811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:48.257357 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:48.257277    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:48.257543 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:48.257276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:48.257543 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:48.257447    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:48.257543 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:48.257507    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:49.256963 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:49.256923    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:49.257378 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:49.257094    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:50.257844 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:50.257810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:50.258299 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:50.257931    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:50.258299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:50.257987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:50.258299 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:50.258090    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:51.257198 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:51.257158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:51.257420 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:51.257294    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:52.257694 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:52.257667    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:52.258002 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:52.257677    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:52.258002 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:52.257775    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:52.258002 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:52.257839    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:53.257222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.257021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:53.257474 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:53.257312    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:53.342573 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.342474    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="2d30ea1d20e0b8850a4192369938cda91a7b7fe8b9fd6e9fe152d83d62578246" exitCode=0
Jul 29 20:47:53.343345 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.342569    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"2d30ea1d20e0b8850a4192369938cda91a7b7fe8b9fd6e9fe152d83d62578246"}
Jul 29 20:47:53.344219 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.344186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zwpjs" event={"ID":"8cbea866-730e-4c2c-b0a2-e16bfb489af8","Type":"ContainerStarted","Data":"f02c96c215c0310462f2c4dbfb68b8a6cc386059600b9f5f24102e06e84f7209"}
Jul 29 20:47:53.346159 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.346130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dl4bj" event={"ID":"f9327713-5426-4824-88bd-2b849ff5997f","Type":"ContainerStarted","Data":"ffd484de13c73102ee9d2cedf5ca9d128be8a4b39658cd2258c336b8a3b0963f"}
Jul 29 20:47:53.347613 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.347582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k2qm7" event={"ID":"9f2bd3e8-d894-4e5a-b11f-c4dc39666d7f","Type":"ContainerStarted","Data":"348d13509cb704937e94b0d10cb5ce4b6cb1b9afe620a8b0997837cf5152d61a"}
Jul 29 20:47:53.349547 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.349511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v9b6d" event={"ID":"4199ec18-e565-4629-bcab-ad9fffdc2aa9","Type":"ContainerStarted","Data":"192a9d8a31bd447cfe1086573644f73fbc547efb236e16de4fd1c4f33b0b9916"}
Jul 29 20:47:53.352910 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.352886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"fdc70b11422c3fb753602d763f894505708329fd01888d2ca569a57c293ac140"}
Jul 29 20:47:53.353012 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.352969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"9158b56d61c81d1aceed0d1745839e7fdce5e70fe24399ea307ccda76045b3bf"}
Jul 29 20:47:53.353012 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.353002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"6b67965aa8c53731305f09fc26fa5da209ca46cd41b1334a7d2c017f25c1aa9e"}
Jul 29 20:47:53.353100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.353031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"40a5c61adf00c5524b09ed3ca8823d7e01455911820507a23aebba674544d907"}
Jul 29 20:47:53.353100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.353064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"58978e1b3b6311b4c3dbf31400bbb61bad5e14d63c2638c2864aaab4606a4390"}
Jul 29 20:47:53.353100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.353097    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"bf704a277b6029a5ad009a5eafeebd3bac71823777561a73c7df1ef0676f560d"}
Jul 29 20:47:53.354362 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.354342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-68k2p" event={"ID":"97471526-67b0-415f-a32a-fdcda1de72ce","Type":"ContainerStarted","Data":"8f0bc0dda7c20d47b905e1a7fc2c1e48365d03784240262ea178a0007ac62d5f"}
Jul 29 20:47:53.355983 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.355959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal" event={"ID":"c882f98f11e192b3a9e8390a2cecc279","Type":"ContainerStarted","Data":"6b9baf392861ba73e40e64ddcf317d2f0e7644e95cf6055b4065e9091c4eabed"}
Jul 29 20:47:53.357453 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.357430    2581 generic.go:356] "Generic (PLEG): container finished" podID="b3dea8349b1cf67101da549d62710210" containerID="898f854da9f03cfbc961568b02c5887963c26611426e52e941e408d82968b02a" exitCode=0
Jul 29 20:47:53.357559 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.357494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal" event={"ID":"b3dea8349b1cf67101da549d62710210","Type":"ContainerDied","Data":"898f854da9f03cfbc961568b02c5887963c26611426e52e941e408d82968b02a"}
Jul 29 20:47:53.359245 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.359224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2" event={"ID":"c1220861-03f4-47ec-9fa7-56b93705461e","Type":"ContainerStarted","Data":"1ca7ee301fdf9ef5f8a7e201915f6a04426583dedd52f3975c1e603c15cb7448"}
Jul 29 20:47:53.379993 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.379949    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k2qm7" podStartSLOduration=2.500237532 podStartE2EDuration="22.379936298s" totalImagesPullingTime="19.879698766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.379741416 +0000 UTC m=+23.628978961" watchObservedRunningTime="2026-07-29 20:47:53.379936298 +0000 UTC m=+23.629173813"
Jul 29 20:47:53.409171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.409116    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dl4bj" podStartSLOduration=2.551224153 podStartE2EDuration="22.40909982s" totalImagesPullingTime="19.857875667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.392782185 +0000 UTC m=+23.642019701" watchObservedRunningTime="2026-07-29 20:47:53.40909982 +0000 UTC m=+23.658337334"
Jul 29 20:47:53.433682 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.433636    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-68k2p" podStartSLOduration=7.553237267 podStartE2EDuration="23.433618085s" totalImagesPullingTime="15.880380818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.409248916 +0000 UTC m=+23.658486430" watchObservedRunningTime="2026-07-29 20:47:53.433618085 +0000 UTC m=+23.682855600"
Jul 29 20:47:53.433842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.433823    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zwpjs" podStartSLOduration=2.569641888 podStartE2EDuration="22.43381737s" totalImagesPullingTime="19.864175482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.433752709 +0000 UTC m=+23.682990207" watchObservedRunningTime="2026-07-29 20:47:53.43381737 +0000 UTC m=+23.683054885"
Jul 29 20:47:53.471561 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.471512    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-229.ec2.internal" podStartSLOduration=21.471495761 podStartE2EDuration="21.471495761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.471024301 +0000 UTC m=+23.720261816" watchObservedRunningTime="2026-07-29 20:47:53.471495761 +0000 UTC m=+23.720733276"
Jul 29 20:47:53.492229 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:53.492168    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v9b6d" podStartSLOduration=2.536548043 podStartE2EDuration="22.492149364s" totalImagesPullingTime="19.955601321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:53.491619421 +0000 UTC m=+23.740856947" watchObservedRunningTime="2026-07-29 20:47:53.492149364 +0000 UTC m=+23.741386881"
Jul 29 20:47:54.021626 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.021395    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 20:47:54.188594 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.188473    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T20:47:54.021619367Z","UUID":"fedaeccc-cddc-40da-8a42-fb4965b47685","Handler":null,"Name":"","Endpoint":""}
Jul 29 20:47:54.190436 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.190410    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 20:47:54.190566 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.190457    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 20:47:54.257593 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.257503    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:54.257593 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.257553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:54.257815 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:54.257639    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:54.258034 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:54.257985    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:54.364323 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.364284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal" event={"ID":"b3dea8349b1cf67101da549d62710210","Type":"ContainerStarted","Data":"8d6d9520afb78fff2b983a5e2b4e7cd7fc11ff962e358071757ca3b56f36bd71"}
Jul 29 20:47:54.366398 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.366346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2" event={"ID":"c1220861-03f4-47ec-9fa7-56b93705461e","Type":"ContainerStarted","Data":"da1105afdf125c749a803655944f1d11790ec19631e8ab777828613dc7e4206c"}
Jul 29 20:47:54.368112 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.368087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8xpd" event={"ID":"ca19729a-23ec-47a7-8cc5-279953c8869f","Type":"ContainerStarted","Data":"0072481dee29441e10ec11f9cb9acb8cd9aeedb6cb18ff50d9fd8fe8bc637562"}
Jul 29 20:47:54.379429 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.379366    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-229.ec2.internal" podStartSLOduration=22.379350642 podStartE2EDuration="22.379350642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:54.379135834 +0000 UTC m=+24.628373349" watchObservedRunningTime="2026-07-29 20:47:54.379350642 +0000 UTC m=+24.628588158"
Jul 29 20:47:54.392332 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:54.392278    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s8xpd" podStartSLOduration=3.497166991 podStartE2EDuration="23.39226427s" totalImagesPullingTime="19.895097279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:54.392258004 +0000 UTC m=+24.641495520" watchObservedRunningTime="2026-07-29 20:47:54.39226427 +0000 UTC m=+24.641501785"
Jul 29 20:47:55.257214 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.257188    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:55.257354 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:55.257321    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:55.342805 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.342772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:55.343378 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.343347    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:55.375121 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.375080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"136bfc9dd87218d852f73fb6e6a3a2b5be8b33e143716151d677e06f6ae1c48d"}
Jul 29 20:47:55.377466 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.377436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2" event={"ID":"c1220861-03f4-47ec-9fa7-56b93705461e","Type":"ContainerStarted","Data":"86515eeb52c70c9d674291e4d6b6f67dbcefa2861e80588414556e945d0d6910"}
Jul 29 20:47:55.378314 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.378291    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:55.378597 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.378582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-68k2p"
Jul 29 20:47:55.396725 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:55.396630    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wkh2" podStartSLOduration=2.218966581 podStartE2EDuration="24.39661337s" totalImagesPullingTime="22.177646789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:55.39660321 +0000 UTC m=+25.645840724" watchObservedRunningTime="2026-07-29 20:47:55.39661337 +0000 UTC m=+25.645850882"
Jul 29 20:47:56.257035 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:56.257003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:56.257035 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:56.257018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:56.257275 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:56.257144    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:56.257275 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:56.257231    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:57.257645 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:57.257608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:57.258157 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:57.257738    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:58.257754 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.257570    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:58.258342 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.257605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:58.258342 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:58.257869    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:58.258342 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:58.257912    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:58.387125 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.387027    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="1a3768030338639fd80c3bcb708952fc430f5273b602243464c50da88202f54d" exitCode=0
Jul 29 20:47:58.387290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.387118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"1a3768030338639fd80c3bcb708952fc430f5273b602243464c50da88202f54d"}
Jul 29 20:47:58.390802 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.390765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" event={"ID":"c18336fb-f2d2-4f88-99af-124a2f4f1539","Type":"ContainerStarted","Data":"59837a5034f620d252ea6a4f6e1f25569364110ee0dedfe0a3ebf1962b520b98"}
Jul 29 20:47:58.391141 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.391106    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:58.391265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.391176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:58.391265 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.391224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:58.407480 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.407449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:58.407625 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.407568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:47:58.434468 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:58.434414    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw" podStartSLOduration=7.456686036 podStartE2EDuration="27.434375811s" totalImagesPullingTime="19.977689775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:58.434178123 +0000 UTC m=+28.683415639" watchObservedRunningTime="2026-07-29 20:47:58.434375811 +0000 UTC m=+28.683613326"
Jul 29 20:47:59.257197 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.257160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:59.257366 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:59.257283    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:47:59.395956 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.395928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerStarted","Data":"42635d0c5668fbcd6a7aa77752d35b11bdd6690f95cd312021260a0627ef152d"}
Jul 29 20:47:59.747435 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.747201    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h78gc"]
Jul 29 20:47:59.747646 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.747542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:47:59.747729 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:59.747638    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:47:59.748008 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.747981    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldb4s"]
Jul 29 20:47:59.748112 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.748104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:47:59.748216 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:59.748189    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:47:59.759062 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.758948    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j7vmc"]
Jul 29 20:47:59.759062 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:47:59.759049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:47:59.759261 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:47:59.759129    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:48:00.400282 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:00.400252    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="42635d0c5668fbcd6a7aa77752d35b11bdd6690f95cd312021260a0627ef152d" exitCode=0
Jul 29 20:48:00.400675 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:00.400344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"42635d0c5668fbcd6a7aa77752d35b11bdd6690f95cd312021260a0627ef152d"}
Jul 29 20:48:01.257650 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:01.257605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:01.258030 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:01.258012    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:01.258150 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:01.258037    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:48:01.258150 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:01.258123    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:48:01.258150 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:01.258131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:01.258285 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:01.258207    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:48:02.407358 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:02.407266    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="4dccb7a052786b7398ea36e68627b2de53dfb5234e7d63c7a5abcfe3ccbec606" exitCode=0
Jul 29 20:48:02.407358 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:02.407318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"4dccb7a052786b7398ea36e68627b2de53dfb5234e7d63c7a5abcfe3ccbec606"}
Jul 29 20:48:03.256773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.256722    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:03.256773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.256767    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:03.256993 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.256773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:03.256993 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.256896    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:48:03.257075 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.257034    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:48:03.257159 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.257123    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:48:03.985254 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.985222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.985271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:03.985306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985419    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985434    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985442    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985455    2581 projected.go:196] Error preparing data for projected volume kube-api-access-4szfp for pod openshift-network-diagnostics/network-check-target-j7vmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985497    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret podName:ee45771b-8495-4116-981b-9dcf8cffedcd nodeName:}" failed. No retries permitted until 2026-07-29 20:48:35.985475065 +0000 UTC m=+66.234712564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret") pod "global-pull-secret-syncer-h78gc" (UID: "ee45771b-8495-4116-981b-9dcf8cffedcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp podName:e71be444-c996-45d9-80ef-e37652bbb81d nodeName:}" failed. No retries permitted until 2026-07-29 20:48:35.985503522 +0000 UTC m=+66.234741016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4szfp" (UniqueName: "kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp") pod "network-check-target-j7vmc" (UID: "e71be444-c996-45d9-80ef-e37652bbb81d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985562    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:48:03.985749 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:03.985591    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:48:35.985581756 +0000 UTC m=+66.234819268 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:48:05.257097 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:05.257054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:05.257097 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:05.257085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:05.257676 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:05.257100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:05.257676 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:05.257178    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7vmc" podUID="e71be444-c996-45d9-80ef-e37652bbb81d"
Jul 29 20:48:05.257676 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:05.257289    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h78gc" podUID="ee45771b-8495-4116-981b-9dcf8cffedcd"
Jul 29 20:48:05.257676 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:05.257421    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:48:06.113033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.112997    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-229.ec2.internal" event="NodeReady"
Jul 29 20:48:06.113199 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.113191    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 20:48:06.146171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.146097    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-574cb46999-db2ts"]
Jul 29 20:48:06.180097 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.180013    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2fn8c"]
Jul 29 20:48:06.180240 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.180156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.182751 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.182706    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 20:48:06.183222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.183205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ws2kf\""
Jul 29 20:48:06.183222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.183214    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 20:48:06.183375 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.183257    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 20:48:06.190598 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.190560    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 20:48:06.196004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.195983    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w6z99"]
Jul 29 20:48:06.196147 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.196121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.198340 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.198315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 20:48:06.198340 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.198330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qgjxr\""
Jul 29 20:48:06.198523 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.198347    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 20:48:06.218994 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.218962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-574cb46999-db2ts"]
Jul 29 20:48:06.218994 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.219000    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2fn8c"]
Jul 29 20:48:06.219204 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.219019    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w6z99"]
Jul 29 20:48:06.219204 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.219055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.221205 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.221178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 20:48:06.221350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.221207    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 20:48:06.221350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.221266    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5tsgl\""
Jul 29 20:48:06.221350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.221329    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 20:48:06.303116 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303116 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-tmp-dir\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-cert\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whfw\" (UniqueName: \"kubernetes.io/projected/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-kube-api-access-7whfw\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-config-volume\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h67hv\" (UniqueName: \"kubernetes.io/projected/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-kube-api-access-h67hv\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.303670 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.303575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64zhn\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.403993 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.403948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64zhn\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.404136    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.404157    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:06.404171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.404235    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:06.904214127 +0000 UTC m=+37.153451642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.404325    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-tmp-dir\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.404421    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:06.904402478 +0000 UTC m=+37.153639987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-cert\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whfw\" (UniqueName: \"kubernetes.io/projected/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-kube-api-access-7whfw\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.404584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.405016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.405016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-config-volume\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.405016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h67hv\" (UniqueName: \"kubernetes.io/projected/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-kube-api-access-h67hv\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.405016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-tmp-dir\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.405016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.404747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.405259 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.405088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-config-volume\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.405567 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.405541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.409190 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.409168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.409309 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.409203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-cert\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.409714 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.409692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.415461 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.415422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h67hv\" (UniqueName: \"kubernetes.io/projected/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-kube-api-access-h67hv\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.415594 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.415525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whfw\" (UniqueName: \"kubernetes.io/projected/3cd52a78-0bfa-49ca-93e5-7f285a6136b4-kube-api-access-7whfw\") pod \"ingress-canary-w6z99\" (UID: \"3cd52a78-0bfa-49ca-93e5-7f285a6136b4\") " pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.415594 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.415536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.416544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.416526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64zhn\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.528858 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.528779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w6z99"
Jul 29 20:48:06.746399 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.746156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w6z99"]
Jul 29 20:48:06.779889 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:48:06.779814    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd52a78_0bfa_49ca_93e5_7f285a6136b4.slice/crio-ca743c7a8e39a02601957db7136c725aec6350a4e3377f0f7f81cd660a50fe38 WatchSource:0}: Error finding container ca743c7a8e39a02601957db7136c725aec6350a4e3377f0f7f81cd660a50fe38: Status 404 returned error can't find the container with id ca743c7a8e39a02601957db7136c725aec6350a4e3377f0f7f81cd660a50fe38
Jul 29 20:48:06.909491 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.909468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:06.909578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:06.909528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:06.909617 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.909597    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:06.909617 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.909612    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:06.909688 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.909617    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:06.909688 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.909664    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:07.90964818 +0000 UTC m=+38.158885672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:06.909688 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:06.909677    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:07.909671116 +0000 UTC m=+38.158908609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:07.257184 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.257103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:07.257341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.257108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:07.257341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.257232    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:07.259392 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.259364    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 20:48:07.260091 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.260075    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ffmvq\""
Jul 29 20:48:07.260155 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.260126    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 20:48:07.260213 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.260151    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 20:48:07.260213 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.260126    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 20:48:07.260213 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.260158    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vp87c\""
Jul 29 20:48:07.422394 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.422339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w6z99" event={"ID":"3cd52a78-0bfa-49ca-93e5-7f285a6136b4","Type":"ContainerStarted","Data":"ca743c7a8e39a02601957db7136c725aec6350a4e3377f0f7f81cd660a50fe38"}
Jul 29 20:48:07.425443 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.425412    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="a8c152226e5d38decd91320bb0137bbe4cf4f9d13b14e871f24e76ef4a608915" exitCode=0
Jul 29 20:48:07.425587 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.425457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"a8c152226e5d38decd91320bb0137bbe4cf4f9d13b14e871f24e76ef4a608915"}
Jul 29 20:48:07.918612 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.918532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:07.918612 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:07.918602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:07.918832 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:07.918765    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:07.918832 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:07.918772    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:07.918935 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:07.918850    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:09.918826338 +0000 UTC m=+40.168063834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:07.918935 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:07.918780    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:07.918935 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:07.918925    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:09.918906591 +0000 UTC m=+40.168144100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:08.431217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:08.431184    2581 generic.go:356] "Generic (PLEG): container finished" podID="1769ec3c-462f-4fb0-86f9-fcecf9756677" containerID="06a542d501c841c0f18dd248468e31a93ea0bfd992d7d476a322e834ef7c99db" exitCode=0
Jul 29 20:48:08.431886 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:08.431226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerDied","Data":"06a542d501c841c0f18dd248468e31a93ea0bfd992d7d476a322e834ef7c99db"}
Jul 29 20:48:09.435817 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.435653    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w6z99" event={"ID":"3cd52a78-0bfa-49ca-93e5-7f285a6136b4","Type":"ContainerStarted","Data":"7ff8e87c3d3ad7dfed7f52bf9472aecb327548aac02da6128a81e30caac62331"}
Jul 29 20:48:09.439035 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.439009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" event={"ID":"1769ec3c-462f-4fb0-86f9-fcecf9756677","Type":"ContainerStarted","Data":"20d0cdfc4fdf180fa65834eaa60340ac3927de6ccc32d80cbfd9518f243c77a6"}
Jul 29 20:48:09.469417 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.469331    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nxnjd" podStartSLOduration=7.756100912 podStartE2EDuration="38.469308624s" totalImagesPullingTime="30.713207712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:09.467343704 +0000 UTC m=+39.716581219" watchObservedRunningTime="2026-07-29 20:48:09.469308624 +0000 UTC m=+39.718546140"
Jul 29 20:48:09.622623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.622585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:09.734799 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.734715    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:09.814848 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.814821    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:09.937128 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.937086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:09.937279 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:09.937170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:09.937279 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:09.937234    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:09.937279 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:09.937253    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:09.937439 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:09.937304    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:09.937439 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:09.937310    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:13.93729547 +0000 UTC m=+44.186532964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:09.937439 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:09.937358    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:13.937346048 +0000 UTC m=+44.186583541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:10.079900 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:10.079800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:10.457604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:10.457500    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w6z99" podStartSLOduration=1.9189590779999999 podStartE2EDuration="4.457482845s" totalImagesPullingTime="2.538523767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:10.45723253 +0000 UTC m=+40.706470045" watchObservedRunningTime="2026-07-29 20:48:10.457482845 +0000 UTC m=+40.706720405"
Jul 29 20:48:10.842259 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:10.842226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:10.898492 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:10.898428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:11.917995 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:11.917955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:11.994667 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:11.994635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:12.224921 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:12.224831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:12.301740 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:12.301705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:13.966346 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:13.966305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:13.966376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:13.966483    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:13.966503    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:13.966509    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:13.966556    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:21.966541531 +0000 UTC m=+52.215779025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:13.966889 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:13.966571    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:21.966564976 +0000 UTC m=+52.215802469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:15.837959 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:15.837915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:15.889850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:15.889809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:22.027867 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:22.027808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:22.027896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:22.027967    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:22.027987    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:22.028002    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:22.028041    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:38.028025854 +0000 UTC m=+68.277263347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:22.028333 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:22.028054    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:48:38.028048445 +0000 UTC m=+68.277285937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:30.412719 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:30.412684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bm4tw"
Jul 29 20:48:30.571206 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:30.571164    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:30.670749 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:30.670676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:31.606853 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:31.606819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:31.661943 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:31.661910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:31.847428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:31.847375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:31.951293 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:31.951201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:32.013890 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:32.013857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:36.035229 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.035177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:36.035229 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.035246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:48:36.035744 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.035270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:36.037599 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.037580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 20:48:36.037689 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.037671    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 20:48:36.037947 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.037934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 20:48:36.046469 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:36.046447    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 20:48:36.046522 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:36.046516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:49:40.046496202 +0000 UTC m=+130.295733707 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : secret "metrics-daemon-secret" not found
Jul 29 20:48:36.048082 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.048067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 20:48:36.049986 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.049966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45771b-8495-4116-981b-9dcf8cffedcd-original-pull-secret\") pod \"global-pull-secret-syncer-h78gc\" (UID: \"ee45771b-8495-4116-981b-9dcf8cffedcd\") " pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:36.059123 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.059100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szfp\" (UniqueName: \"kubernetes.io/projected/e71be444-c996-45d9-80ef-e37652bbb81d-kube-api-access-4szfp\") pod \"network-check-target-j7vmc\" (UID: \"e71be444-c996-45d9-80ef-e37652bbb81d\") " pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:36.066928 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.066905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h78gc"
Jul 29 20:48:36.080776 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.080753    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ffmvq\""
Jul 29 20:48:36.089532 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.089507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:36.226755 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.226720    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h78gc"]
Jul 29 20:48:36.243415 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.243374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j7vmc"]
Jul 29 20:48:36.246411 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:48:36.246375    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71be444_c996_45d9_80ef_e37652bbb81d.slice/crio-2ca04b096764d0dde884239f63f12e697df38eec688872097e06003abbf0fa96 WatchSource:0}: Error finding container 2ca04b096764d0dde884239f63f12e697df38eec688872097e06003abbf0fa96: Status 404 returned error can't find the container with id 2ca04b096764d0dde884239f63f12e697df38eec688872097e06003abbf0fa96
Jul 29 20:48:36.501189 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.501149    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j7vmc" event={"ID":"e71be444-c996-45d9-80ef-e37652bbb81d","Type":"ContainerStarted","Data":"2ca04b096764d0dde884239f63f12e697df38eec688872097e06003abbf0fa96"}
Jul 29 20:48:36.502235 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:36.502207    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h78gc" event={"ID":"ee45771b-8495-4116-981b-9dcf8cffedcd","Type":"ContainerStarted","Data":"f547803cbb40acda88a19b4a8f41813a9abc798159b8176bd78abb9ae7d4dd30"}
Jul 29 20:48:38.055773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:38.055736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:38.055803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:38.055913    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:38.055938    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:38.055942    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:38.056003    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:49:10.055984726 +0000 UTC m=+100.305222226 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:48:38.056483 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:48:38.056083    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:49:10.056073741 +0000 UTC m=+100.305311235 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:48:40.514907 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:40.514874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j7vmc" event={"ID":"e71be444-c996-45d9-80ef-e37652bbb81d","Type":"ContainerStarted","Data":"fa3f49294c5939b7744310306ac8dab52ac702dea087854da7c260b0d23aae2a"}
Jul 29 20:48:40.515369 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:40.514957    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:48:40.530181 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:40.530116    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j7vmc" podStartSLOduration=67.090531634 podStartE2EDuration="1m10.530097816s" totalImagesPullingTime="3.439566182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:40.529449253 +0000 UTC m=+70.778686784" watchObservedRunningTime="2026-07-29 20:48:40.530097816 +0000 UTC m=+70.779335331"
Jul 29 20:48:42.521034 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:42.520998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h78gc" event={"ID":"ee45771b-8495-4116-981b-9dcf8cffedcd","Type":"ContainerStarted","Data":"a9fd1fe5d9962a0132c3daa3f1fc0f46c99b06f72a68d8a0beccc39b4877c951"}
Jul 29 20:48:45.251565 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:45.251529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:45.310736 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:45.310704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:56.660420 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:56.660267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:56.711506 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:48:56.711467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:10.085129 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:49:10.085084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:49:10.085146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:10.085235    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:10.085234    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:10.085256    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:10.085295    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:50:14.085281862 +0000 UTC m=+164.334519355 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:49:10.085575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:10.085312    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:50:14.085300455 +0000 UTC m=+164.334537949 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:49:11.520874 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:49:11.520844    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j7vmc"
Jul 29 20:49:11.535449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:49:11.535372    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h78gc" podStartSLOduration=94.699482095 podStartE2EDuration="1m40.535356098s" totalImagesPullingTime="5.835874003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:42.53575865 +0000 UTC m=+72.784996165" watchObservedRunningTime="2026-07-29 20:49:11.535356098 +0000 UTC m=+101.784593613"
Jul 29 20:49:40.086305 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:49:40.086250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:49:40.086819 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:40.086423    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 20:49:40.086819 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:49:40.086488    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:51:42.086472167 +0000 UTC m=+252.335709660 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : secret "metrics-daemon-secret" not found
Jul 29 20:50:09.191742 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:09.191690    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-574cb46999-db2ts" podUID="efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"
Jul 29 20:50:09.206888 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:09.206847    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2fn8c" podUID="80a2cb8b-db33-4b5a-84c7-8e18d332a1d5"
Jul 29 20:50:09.725905 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:50:09.725880    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:50:09.726074 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:50:09.725914    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:50:10.273131 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:10.273094    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:50:14.108854 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:50:14.108805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") pod \"image-registry-574cb46999-db2ts\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") " pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:50:14.108879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:14.108970    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:14.108987    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574cb46999-db2ts: secret "image-registry-tls" not found
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:14.108987    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:14.109048    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls podName:80a2cb8b-db33-4b5a-84c7-8e18d332a1d5 nodeName:}" failed. No retries permitted until 2026-07-29 20:52:16.109032518 +0000 UTC m=+286.358270010 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls") pod "dns-default-2fn8c" (UID: "80a2cb8b-db33-4b5a-84c7-8e18d332a1d5") : secret "dns-default-metrics-tls" not found
Jul 29 20:50:14.109254 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:50:14.109063    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls podName:efa1a824-7b5c-4a10-bfed-c59adbfdc2c8 nodeName:}" failed. No retries permitted until 2026-07-29 20:52:16.109057098 +0000 UTC m=+286.358294591 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls") pod "image-registry-574cb46999-db2ts" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8") : secret "image-registry-tls" not found
Jul 29 20:50:23.257047 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:50:23.256939    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:51:39.532942 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.532900    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"]
Jul 29 20:51:39.534969 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.534953    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"]
Jul 29 20:51:39.535111 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.535085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"
Jul 29 20:51:39.536646 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.536625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.537197 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.537182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wrdmr\""
Jul 29 20:51:39.540254 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.540185    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 20:51:39.540405 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.540200    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:51:39.540486 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.540450    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-869qz\""
Jul 29 20:51:39.540570 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.540503    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 20:51:39.540570 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.540528    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 20:51:39.542368 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.542349    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"]
Jul 29 20:51:39.544070 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.544049    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"]
Jul 29 20:51:39.544168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.544157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.545611 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.545595    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5799b9b4f7-h2vnm"]
Jul 29 20:51:39.545705 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.545691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"
Jul 29 20:51:39.546252 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.546234    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 20:51:39.546339 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.546253    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 20:51:39.546413 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.546355    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:51:39.546465 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.546436    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f9ss4\""
Jul 29 20:51:39.547150 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.547132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.547654 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.547635    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:51:39.547745 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.547728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h5mbk\""
Jul 29 20:51:39.547793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.547750    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 20:51:39.549622 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.549605    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gbpw8\""
Jul 29 20:51:39.549710 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.549633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 20:51:39.549905 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.549890    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 20:51:39.549976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.549946    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 20:51:39.550080 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.550064    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 20:51:39.550147 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.550133    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 20:51:39.550225 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.550212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 20:51:39.558659 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.558637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"]
Jul 29 20:51:39.559398 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.559362    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"]
Jul 29 20:51:39.563347 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.563329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"]
Jul 29 20:51:39.564444 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.564426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5799b9b4f7-h2vnm"]
Jul 29 20:51:39.570420 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.570367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"]
Jul 29 20:51:39.637933 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.637903    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9v96z"]
Jul 29 20:51:39.639822 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.639805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.642192 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.642153    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:51:39.642192 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.642180    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 20:51:39.642341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.642180    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wbwwm\""
Jul 29 20:51:39.642341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.642221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 20:51:39.642341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.642153    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 20:51:39.648605 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.648581    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 20:51:39.651081 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.651057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9v96z"]
Jul 29 20:51:39.719033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.718994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng5ls\" (UniqueName: \"kubernetes.io/projected/0e8fc6fc-b42b-40a8-8b52-e9744772bd69-kube-api-access-ng5ls\") pod \"network-check-source-7d88cb55d8-p7gcw\" (UID: \"0e8fc6fc-b42b-40a8-8b52-e9744772bd69\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"
Jul 29 20:51:39.719033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-stats-auth\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a802bb-5da0-4c97-b698-997813f976a3-config\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rf7\" (UniqueName: \"kubernetes.io/projected/52d152d2-dcf1-46bd-a446-b0de2a577e7a-kube-api-access-m2rf7\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whb69\" (UniqueName: \"kubernetes.io/projected/f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b-kube-api-access-whb69\") pod \"volume-data-source-validator-549c6d54cf-tlhzp\" (UID: \"f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-default-certificate\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a802bb-5da0-4c97-b698-997813f976a3-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.719299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.719552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4l92\" (UniqueName: \"kubernetes.io/projected/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-kube-api-access-s4l92\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.719552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.719552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.719433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjpjm\" (UniqueName: \"kubernetes.io/projected/12a802bb-5da0-4c97-b698-997813f976a3-kube-api-access-wjpjm\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.748469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.748434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:39.820143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4l92\" (UniqueName: \"kubernetes.io/projected/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-kube-api-access-s4l92\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.820143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.820143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng5ls\" (UniqueName: \"kubernetes.io/projected/0e8fc6fc-b42b-40a8-8b52-e9744772bd69-kube-api-access-ng5ls\") pod \"network-check-source-7d88cb55d8-p7gcw\" (UID: \"0e8fc6fc-b42b-40a8-8b52-e9744772bd69\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-stats-auth\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-default-certificate\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a802bb-5da0-4c97-b698-997813f976a3-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjpjm\" (UniqueName: \"kubernetes.io/projected/12a802bb-5da0-4c97-b698-997813f976a3-kube-api-access-wjpjm\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.820434 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:39.820403    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b0866d-ea8c-487d-9cec-9b85d83a185e-serving-cert\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:39.820407    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:39.820494    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls podName:52d152d2-dcf1-46bd-a446-b0de2a577e7a nodeName:}" failed. No retries permitted until 2026-07-29 20:51:40.320463184 +0000 UTC m=+250.569700690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls") pod "cluster-samples-operator-6748489656-tdsbl" (UID: "52d152d2-dcf1-46bd-a446-b0de2a577e7a") : secret "samples-operator-tls" not found
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:39.820551    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:40.320537569 +0000 UTC m=+250.569775062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : secret "router-metrics-certs-default" not found
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-config\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a802bb-5da0-4c97-b698-997813f976a3-config\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rf7\" (UniqueName: \"kubernetes.io/projected/52d152d2-dcf1-46bd-a446-b0de2a577e7a-kube-api-access-m2rf7\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.820722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whb69\" (UniqueName: \"kubernetes.io/projected/f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b-kube-api-access-whb69\") pod \"volume-data-source-validator-549c6d54cf-tlhzp\" (UID: \"f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"
Jul 29 20:51:39.821093 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nvn5\" (UniqueName: \"kubernetes.io/projected/a8b0866d-ea8c-487d-9cec-9b85d83a185e-kube-api-access-2nvn5\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.821093 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.821093 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.820858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-trusted-ca\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.821093 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:39.820975    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:40.320958975 +0000 UTC m=+250.570196484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : configmap references non-existent config key: service-ca.crt
Jul 29 20:51:39.821377 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.821354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a802bb-5da0-4c97-b698-997813f976a3-config\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.822806 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.822787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a802bb-5da0-4c97-b698-997813f976a3-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.822937 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.822922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-default-certificate\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.822979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.822959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-stats-auth\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.830774 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.830749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whb69\" (UniqueName: \"kubernetes.io/projected/f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b-kube-api-access-whb69\") pod \"volume-data-source-validator-549c6d54cf-tlhzp\" (UID: \"f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"
Jul 29 20:51:39.830970 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.830951    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4l92\" (UniqueName: \"kubernetes.io/projected/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-kube-api-access-s4l92\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:39.831243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.831211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng5ls\" (UniqueName: \"kubernetes.io/projected/0e8fc6fc-b42b-40a8-8b52-e9744772bd69-kube-api-access-ng5ls\") pod \"network-check-source-7d88cb55d8-p7gcw\" (UID: \"0e8fc6fc-b42b-40a8-8b52-e9744772bd69\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"
Jul 29 20:51:39.831326 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.831281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjpjm\" (UniqueName: \"kubernetes.io/projected/12a802bb-5da0-4c97-b698-997813f976a3-kube-api-access-wjpjm\") pod \"service-ca-operator-5d6c75f84f-2nmb7\" (UID: \"12a802bb-5da0-4c97-b698-997813f976a3\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.831578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.831559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rf7\" (UniqueName: \"kubernetes.io/projected/52d152d2-dcf1-46bd-a446-b0de2a577e7a-kube-api-access-m2rf7\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:39.835729 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.835703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:39.845530 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.845511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"
Jul 29 20:51:39.851779 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.851760    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"
Jul 29 20:51:39.865553 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.865526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"
Jul 29 20:51:39.922128 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.921913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b0866d-ea8c-487d-9cec-9b85d83a185e-serving-cert\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.922128 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.921957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-config\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.922355 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.922147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nvn5\" (UniqueName: \"kubernetes.io/projected/a8b0866d-ea8c-487d-9cec-9b85d83a185e-kube-api-access-2nvn5\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.922355 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.922203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-trusted-ca\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.923056 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.923006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-config\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.923484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.923327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b0866d-ea8c-487d-9cec-9b85d83a185e-trusted-ca\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.926710 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.926657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b0866d-ea8c-487d-9cec-9b85d83a185e-serving-cert\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.931963 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.931936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nvn5\" (UniqueName: \"kubernetes.io/projected/a8b0866d-ea8c-487d-9cec-9b85d83a185e-kube-api-access-2nvn5\") pod \"console-operator-584dcb6b54-9v96z\" (UID: \"a8b0866d-ea8c-487d-9cec-9b85d83a185e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.949946 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.949741    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:39.993900 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:39.993792    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw"]
Jul 29 20:51:39.997740 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:39.997695    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e8fc6fc_b42b_40a8_8b52_e9744772bd69.slice/crio-38b378057ff5877bd382192bc92320036b1afea4d3a6577359aeaba512a19f6a WatchSource:0}: Error finding container 38b378057ff5877bd382192bc92320036b1afea4d3a6577359aeaba512a19f6a: Status 404 returned error can't find the container with id 38b378057ff5877bd382192bc92320036b1afea4d3a6577359aeaba512a19f6a
Jul 29 20:51:40.012411 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.012350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7"]
Jul 29 20:51:40.034466 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.034439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp"]
Jul 29 20:51:40.037495 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:40.037466    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ffb7a0_6bfd_43bf_8f96_6192cfeab75b.slice/crio-c0d1eb728ecb387a7d8e44d527fd699534d2492da7df39e77567a88e9ca3e46d WatchSource:0}: Error finding container c0d1eb728ecb387a7d8e44d527fd699534d2492da7df39e77567a88e9ca3e46d: Status 404 returned error can't find the container with id c0d1eb728ecb387a7d8e44d527fd699534d2492da7df39e77567a88e9ca3e46d
Jul 29 20:51:40.091612 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.091534    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9v96z"]
Jul 29 20:51:40.095048 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:40.095020    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b0866d_ea8c_487d_9cec_9b85d83a185e.slice/crio-0ff5272d0e8928e8500a77b13d3d0b6d2bd276ef54ffc80e475f25d4021be02c WatchSource:0}: Error finding container 0ff5272d0e8928e8500a77b13d3d0b6d2bd276ef54ffc80e475f25d4021be02c: Status 404 returned error can't find the container with id 0ff5272d0e8928e8500a77b13d3d0b6d2bd276ef54ffc80e475f25d4021be02c
Jul 29 20:51:40.325727 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.325689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:40.325727 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.325734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:40.325916 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.325777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:40.325916 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:40.325868    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 20:51:40.325916 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:40.325882    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:41.325862629 +0000 UTC m=+251.575100123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : configmap references non-existent config key: service-ca.crt
Jul 29 20:51:40.325916 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:40.325904    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:41.325893155 +0000 UTC m=+251.575130648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : secret "router-metrics-certs-default" not found
Jul 29 20:51:40.326081 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:40.325931    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 20:51:40.326081 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:40.325989    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls podName:52d152d2-dcf1-46bd-a446-b0de2a577e7a nodeName:}" failed. No retries permitted until 2026-07-29 20:51:41.325974005 +0000 UTC m=+251.575211501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls") pod "cluster-samples-operator-6748489656-tdsbl" (UID: "52d152d2-dcf1-46bd-a446-b0de2a577e7a") : secret "samples-operator-tls" not found
Jul 29 20:51:40.938343 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.938257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp" event={"ID":"f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b","Type":"ContainerStarted","Data":"c0d1eb728ecb387a7d8e44d527fd699534d2492da7df39e77567a88e9ca3e46d"}
Jul 29 20:51:40.940353 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.940271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7" event={"ID":"12a802bb-5da0-4c97-b698-997813f976a3","Type":"ContainerStarted","Data":"9de6c6184222e7246b5713fbbc9ebc926d194e2550d016b2dd36f62444689ab9"}
Jul 29 20:51:40.942442 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.942415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" event={"ID":"a8b0866d-ea8c-487d-9cec-9b85d83a185e","Type":"ContainerStarted","Data":"0ff5272d0e8928e8500a77b13d3d0b6d2bd276ef54ffc80e475f25d4021be02c"}
Jul 29 20:51:40.944526 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.944503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw" event={"ID":"0e8fc6fc-b42b-40a8-8b52-e9744772bd69","Type":"ContainerStarted","Data":"ea6453eacd2dc9242b65c63c8c221358c097fa89857ac47efc63686fa7f0d6d7"}
Jul 29 20:51:40.944607 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.944544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw" event={"ID":"0e8fc6fc-b42b-40a8-8b52-e9744772bd69","Type":"ContainerStarted","Data":"38b378057ff5877bd382192bc92320036b1afea4d3a6577359aeaba512a19f6a"}
Jul 29 20:51:40.963590 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:40.963166    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-p7gcw" podStartSLOduration=1.9631240349999999 podStartE2EDuration="1.963124035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:40.961200565 +0000 UTC m=+251.210438083" watchObservedRunningTime="2026-07-29 20:51:40.963124035 +0000 UTC m=+251.212361545"
Jul 29 20:51:41.333783 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:41.333545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:41.333783 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:41.333604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:41.333783 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:41.333656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:41.333783 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:41.333778    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 20:51:41.334117 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:41.333836    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:43.333820359 +0000 UTC m=+253.583057864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : secret "router-metrics-certs-default" not found
Jul 29 20:51:41.334371 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:41.334246    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:43.334229517 +0000 UTC m=+253.583467022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : configmap references non-existent config key: service-ca.crt
Jul 29 20:51:41.334371 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:41.334325    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 20:51:41.334371 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:41.334359    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls podName:52d152d2-dcf1-46bd-a446-b0de2a577e7a nodeName:}" failed. No retries permitted until 2026-07-29 20:51:43.334349056 +0000 UTC m=+253.583586553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls") pod "cluster-samples-operator-6748489656-tdsbl" (UID: "52d152d2-dcf1-46bd-a446-b0de2a577e7a") : secret "samples-operator-tls" not found
Jul 29 20:51:41.949302 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:41.949267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp" event={"ID":"f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b","Type":"ContainerStarted","Data":"9a432908a32c4da0b5818199527e1f304b1a9e14f0e8c2498ca60d096e443c5e"}
Jul 29 20:51:41.965752 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:41.965690    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tlhzp" podStartSLOduration=1.574761555 podStartE2EDuration="2.965671182s" totalImagesPullingTime="1.390909627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:41.96529616 +0000 UTC m=+252.214533675" watchObservedRunningTime="2026-07-29 20:51:41.965671182 +0000 UTC m=+252.214908695"
Jul 29 20:51:42.040585 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.040549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:42.139820 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.139782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:51:42.140008 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:42.139936    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 20:51:42.140071 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:42.140009    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs podName:8b7f12a9-0f65-492a-8764-a3d0ace547fe nodeName:}" failed. No retries permitted until 2026-07-29 20:53:44.139990915 +0000 UTC m=+374.389228409 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs") pod "network-metrics-daemon-ldb4s" (UID: "8b7f12a9-0f65-492a-8764-a3d0ace547fe") : secret "metrics-daemon-secret" not found
Jul 29 20:51:42.141715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.141683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:42.954561 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.954537    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/0.log"
Jul 29 20:51:42.954952 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.954572    2581 generic.go:356] "Generic (PLEG): container finished" podID="a8b0866d-ea8c-487d-9cec-9b85d83a185e" containerID="50f351531bd1003563cd74240bd8a42d33f7ae6704dcfe83eb350b11996448b0" exitCode=255
Jul 29 20:51:42.954952 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.954604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" event={"ID":"a8b0866d-ea8c-487d-9cec-9b85d83a185e","Type":"ContainerDied","Data":"50f351531bd1003563cd74240bd8a42d33f7ae6704dcfe83eb350b11996448b0"}
Jul 29 20:51:42.954952 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.954849    2581 scope.go:117] "RemoveContainer" containerID="50f351531bd1003563cd74240bd8a42d33f7ae6704dcfe83eb350b11996448b0"
Jul 29 20:51:42.956267 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.956233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7" event={"ID":"12a802bb-5da0-4c97-b698-997813f976a3","Type":"ContainerStarted","Data":"affde5860208d6245cf9496bd0cda059a568bbc25b13e4bf62b4fe3377e7b0ff"}
Jul 29 20:51:42.983997 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:42.983947    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7" podStartSLOduration=1.191188675 podStartE2EDuration="3.983928594s" totalImagesPullingTime="2.792739919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:42.98312007 +0000 UTC m=+253.232357584" watchObservedRunningTime="2026-07-29 20:51:42.983928594 +0000 UTC m=+253.233166110"
Jul 29 20:51:43.352336 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.352203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:43.352336 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.352330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:43.352575 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.352376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:43.352575 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.352449    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:47.352420537 +0000 UTC m=+257.601658048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : configmap references non-existent config key: service-ca.crt
Jul 29 20:51:43.352688 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.352602    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 20:51:43.352985 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.352966    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 20:51:43.353036 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.353003    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls podName:52d152d2-dcf1-46bd-a446-b0de2a577e7a nodeName:}" failed. No retries permitted until 2026-07-29 20:51:47.352985733 +0000 UTC m=+257.602223242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls") pod "cluster-samples-operator-6748489656-tdsbl" (UID: "52d152d2-dcf1-46bd-a446-b0de2a577e7a") : secret "samples-operator-tls" not found
Jul 29 20:51:43.353036 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.353027    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:47.353012304 +0000 UTC m=+257.602249816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : secret "router-metrics-certs-default" not found
Jul 29 20:51:43.361439 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.361417    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p97db"]
Jul 29 20:51:43.363504 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.363488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.365487 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.365469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 20:51:43.365602 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.365584    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wqct\""
Jul 29 20:51:43.365865 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.365849    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 20:51:43.366209 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.366196    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 20:51:43.366275 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.366214    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 20:51:43.388504 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.388479    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p97db"]
Jul 29 20:51:43.431976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.431939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:43.454028 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.453991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5787257-a1af-438f-80f8-b503a6463853-data-volume\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.454028 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.454029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvzz7\" (UniqueName: \"kubernetes.io/projected/f5787257-a1af-438f-80f8-b503a6463853-kube-api-access-dvzz7\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.454223 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.454054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.454223 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.454098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.454223 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.454166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5787257-a1af-438f-80f8-b503a6463853-crio-socket\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.487396 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.487348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:43.555017 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.554982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5787257-a1af-438f-80f8-b503a6463853-crio-socket\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5787257-a1af-438f-80f8-b503a6463853-data-volume\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvzz7\" (UniqueName: \"kubernetes.io/projected/f5787257-a1af-438f-80f8-b503a6463853-kube-api-access-dvzz7\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555208 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5787257-a1af-438f-80f8-b503a6463853-crio-socket\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.555476 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.555227    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:43.555476 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.555224    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:51:43.555476 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.555286    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:44.055268574 +0000 UTC m=+254.304506067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:43.555476 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.555302    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:44.055293924 +0000 UTC m=+254.304531417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : configmap "kube-rbac-proxy" not found
Jul 29 20:51:43.555476 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.555425    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5787257-a1af-438f-80f8-b503a6463853-data-volume\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.563037 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.563009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvzz7\" (UniqueName: \"kubernetes.io/projected/f5787257-a1af-438f-80f8-b503a6463853-kube-api-access-dvzz7\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:43.961069 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961044    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:51:43.961517 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961500    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/0.log"
Jul 29 20:51:43.961562 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961542    2581 generic.go:356] "Generic (PLEG): container finished" podID="a8b0866d-ea8c-487d-9cec-9b85d83a185e" containerID="85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658" exitCode=255
Jul 29 20:51:43.961630 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" event={"ID":"a8b0866d-ea8c-487d-9cec-9b85d83a185e","Type":"ContainerDied","Data":"85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658"}
Jul 29 20:51:43.961676 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961664    2581 scope.go:117] "RemoveContainer" containerID="50f351531bd1003563cd74240bd8a42d33f7ae6704dcfe83eb350b11996448b0"
Jul 29 20:51:43.961876 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:43.961855    2581 scope.go:117] "RemoveContainer" containerID="85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658"
Jul 29 20:51:43.962073 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:43.962047    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9v96z_openshift-console-operator(a8b0866d-ea8c-487d-9cec-9b85d83a185e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" podUID="a8b0866d-ea8c-487d-9cec-9b85d83a185e"
Jul 29 20:51:44.059683 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.059288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:44.059683 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.059371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:44.059902 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:44.059698    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:44.059902 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:44.059778    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:45.059753986 +0000 UTC m=+255.308991497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:44.060150 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:44.060126    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:51:44.060301 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:44.060190    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:45.060174795 +0000 UTC m=+255.309412304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : configmap "kube-rbac-proxy" not found
Jul 29 20:51:44.116720 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.116684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:44.219257 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.219139    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:44.320540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.320497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:44.911465 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.911430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:44.965814 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.965790    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:51:44.966152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.966091    2581 scope.go:117] "RemoveContainer" containerID="85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658"
Jul 29 20:51:44.966255 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:44.966237    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9v96z_openshift-console-operator(a8b0866d-ea8c-487d-9cec-9b85d83a185e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" podUID="a8b0866d-ea8c-487d-9cec-9b85d83a185e"
Jul 29 20:51:44.996890 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:44.996861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:45.068397 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:45.068343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:45.068551 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:45.068473    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:51:45.068551 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:45.068531    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:47.068518315 +0000 UTC m=+257.317755808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : configmap "kube-rbac-proxy" not found
Jul 29 20:51:45.068551 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:45.068527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:45.068679 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:45.068592    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:45.068679 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:45.068641    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:47.068626967 +0000 UTC m=+257.317864465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:45.178011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:45.177945    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dl4bj_f9327713-5426-4824-88bd-2b849ff5997f/dns-node-resolver/0.log"
Jul 29 20:51:46.056425 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.056373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.157412 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.157347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.504957 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.504927    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7th82"]
Jul 29 20:51:46.507649 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.507632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.509800 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.509778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 20:51:46.509889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.509778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 20:51:46.510543 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.510529    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 20:51:46.510640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.510571    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 20:51:46.510640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.510577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vbfhn\""
Jul 29 20:51:46.516699 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.516676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7th82"]
Jul 29 20:51:46.539930 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.539888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.578350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.578329    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zwpjs_8cbea866-730e-4c2c-b0a2-e16bfb489af8/node-ca/0.log"
Jul 29 20:51:46.581649 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.581629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/665e6009-541a-412c-bc83-2db529bb8ba3-signing-cabundle\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.581773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.581670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gsm\" (UniqueName: \"kubernetes.io/projected/665e6009-541a-412c-bc83-2db529bb8ba3-kube-api-access-v9gsm\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.581773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.581736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/665e6009-541a-412c-bc83-2db529bb8ba3-signing-key\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.594308 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.594278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.682927 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.682894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gsm\" (UniqueName: \"kubernetes.io/projected/665e6009-541a-412c-bc83-2db529bb8ba3-kube-api-access-v9gsm\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.683063 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.682950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/665e6009-541a-412c-bc83-2db529bb8ba3-signing-key\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.683063 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.683037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/665e6009-541a-412c-bc83-2db529bb8ba3-signing-cabundle\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.683792 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.683775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/665e6009-541a-412c-bc83-2db529bb8ba3-signing-cabundle\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.685590 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.685576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/665e6009-541a-412c-bc83-2db529bb8ba3-signing-key\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.691904 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.691880    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gsm\" (UniqueName: \"kubernetes.io/projected/665e6009-541a-412c-bc83-2db529bb8ba3-kube-api-access-v9gsm\") pod \"service-ca-658d7b7c78-7th82\" (UID: \"665e6009-541a-412c-bc83-2db529bb8ba3\") " pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.816822 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.816730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7th82"
Jul 29 20:51:46.938865 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.938831    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7th82"]
Jul 29 20:51:46.945531 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:46.945497    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod665e6009_541a_412c_bc83_2db529bb8ba3.slice/crio-0562a922bff78d850ca729b595892d385194dd0268a20e51dee90c0363acab70 WatchSource:0}: Error finding container 0562a922bff78d850ca729b595892d385194dd0268a20e51dee90c0363acab70: Status 404 returned error can't find the container with id 0562a922bff78d850ca729b595892d385194dd0268a20e51dee90c0363acab70
Jul 29 20:51:46.974038 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.974010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7th82" event={"ID":"665e6009-541a-412c-bc83-2db529bb8ba3","Type":"ContainerStarted","Data":"0562a922bff78d850ca729b595892d385194dd0268a20e51dee90c0363acab70"}
Jul 29 20:51:46.978465 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:46.978445    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w6z99_3cd52a78-0bfa-49ca-93e5-7f285a6136b4/serve-healthcheck-canary/0.log"
Jul 29 20:51:47.086575 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.086481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:47.086575 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.086533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:47.086982 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.086601    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:51:47.086982 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.086639    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:47.086982 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.086669    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:51.086654294 +0000 UTC m=+261.335891786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : configmap "kube-rbac-proxy" not found
Jul 29 20:51:47.086982 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.086683    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:51.086677305 +0000 UTC m=+261.335914797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:47.389710 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.389622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:47.389710 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.389672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.389725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.389796    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:55.389775642 +0000 UTC m=+265.639013154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : configmap references non-existent config key: service-ca.crt
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.389839    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.389840    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.389889    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls podName:52d152d2-dcf1-46bd-a446-b0de2a577e7a nodeName:}" failed. No retries permitted until 2026-07-29 20:51:55.389877123 +0000 UTC m=+265.639114615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls") pod "cluster-samples-operator-6748489656-tdsbl" (UID: "52d152d2-dcf1-46bd-a446-b0de2a577e7a") : secret "samples-operator-tls" not found
Jul 29 20:51:47.389934 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:47.389918    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs podName:3ea4a380-85fe-4581-9a06-e49d7dc8ca61 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:55.3899123 +0000 UTC m=+265.639149792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs") pod "router-default-5799b9b4f7-h2vnm" (UID: "3ea4a380-85fe-4581-9a06-e49d7dc8ca61") : secret "router-metrics-certs-default" not found
Jul 29 20:51:47.423879 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.423847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:47.978905 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.978863    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7th82" event={"ID":"665e6009-541a-412c-bc83-2db529bb8ba3","Type":"ContainerStarted","Data":"b7904d386b3728ceaed17798f2928c93a4b8b6d28e5c62643ee7540c898bf22b"}
Jul 29 20:51:47.994850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:47.994795    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-7th82" podStartSLOduration=1.994780733 podStartE2EDuration="1.994780733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:47.994742738 +0000 UTC m=+258.243980256" watchObservedRunningTime="2026-07-29 20:51:47.994780733 +0000 UTC m=+258.244018247"
Jul 29 20:51:48.446327 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:48.446295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:48.792151 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:48.792084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:48.891809 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:48.891774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:48.991828 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:48.991792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:49.156057 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.155957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:49.258943 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.258914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:49.769359 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.769315    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:49.950526 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.950451    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:49.950526 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.950501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:51:49.950816 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:49.950803    2581 scope.go:117] "RemoveContainer" containerID="85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658"
Jul 29 20:51:49.950963 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:49.950947    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9v96z_openshift-console-operator(a8b0866d-ea8c-487d-9cec-9b85d83a185e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" podUID="a8b0866d-ea8c-487d-9cec-9b85d83a185e"
Jul 29 20:51:50.177627 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:50.177575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:50.556827 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:50.556786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:50.771535 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:50.771459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:51.120442 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.120376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:51.120646 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:51.120466    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:51:51.120646 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:51.120526    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:59.120510622 +0000 UTC m=+269.369748114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : configmap "kube-rbac-proxy" not found
Jul 29 20:51:51.120646 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.120465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:51.120646 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:51.120547    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:51.120646 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:51:51.120600    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls podName:f5787257-a1af-438f-80f8-b503a6463853 nodeName:}" failed. No retries permitted until 2026-07-29 20:51:59.12058628 +0000 UTC m=+269.369823796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p97db" (UID: "f5787257-a1af-438f-80f8-b503a6463853") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:51:51.184570 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.184529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:51.353962 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.353922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:51.454584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.454485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:51.769463 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:51.769431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:52.334744 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:52.334695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:52.392505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:52.392473    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:52.735244 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:52.735135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:52.790571 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:52.790538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:52.968353 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:52.968317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:53.567801 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:53.567757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:53.623238 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:53.623204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:53.909626 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:53.909545    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:54.170934 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:54.170843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:54.363969 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:54.363911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:54.577402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:54.577362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:54.967462 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:54.967352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:55.456438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.456408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:55.456632 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.456447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:55.456632 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.456480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:55.457013 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.456990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-service-ca-bundle\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:55.458940 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.458918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ea4a380-85fe-4581-9a06-e49d7dc8ca61-metrics-certs\") pod \"router-default-5799b9b4f7-h2vnm\" (UID: \"3ea4a380-85fe-4581-9a06-e49d7dc8ca61\") " pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:55.459037 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.458988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/52d152d2-dcf1-46bd-a446-b0de2a577e7a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tdsbl\" (UID: \"52d152d2-dcf1-46bd-a446-b0de2a577e7a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:55.471843 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.471825    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:55.597645 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.597618    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5799b9b4f7-h2vnm"]
Jul 29 20:51:55.600291 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:55.600266    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea4a380_85fe_4581_9a06_e49d7dc8ca61.slice/crio-91a145ca1293b535feed82e7c84418c84f17ae981d930e1386ad7a2fcdb21550 WatchSource:0}: Error finding container 91a145ca1293b535feed82e7c84418c84f17ae981d930e1386ad7a2fcdb21550: Status 404 returned error can't find the container with id 91a145ca1293b535feed82e7c84418c84f17ae981d930e1386ad7a2fcdb21550
Jul 29 20:51:55.757753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.757718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"
Jul 29 20:51:55.773655 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.773627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:55.885077 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.885043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl"]
Jul 29 20:51:55.999778 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:55.999745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.004591 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.004563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm" event={"ID":"3ea4a380-85fe-4581-9a06-e49d7dc8ca61","Type":"ContainerStarted","Data":"b7de36d562098100706a7f4589cd47844e83d8ac63b5bb1f237631b938f3be2a"}
Jul 29 20:51:56.004722 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.004616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm" event={"ID":"3ea4a380-85fe-4581-9a06-e49d7dc8ca61","Type":"ContainerStarted","Data":"91a145ca1293b535feed82e7c84418c84f17ae981d930e1386ad7a2fcdb21550"}
Jul 29 20:51:56.005698 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.005674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl" event={"ID":"52d152d2-dcf1-46bd-a446-b0de2a577e7a","Type":"ContainerStarted","Data":"010fa499a0328e10b10fc78a02dfad6e2c72d0d20026e8fecaecac19e0794d32"}
Jul 29 20:51:56.024352 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.024266    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm" podStartSLOduration=17.024247176 podStartE2EDuration="17.024247176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:56.022047165 +0000 UTC m=+266.271284683" watchObservedRunningTime="2026-07-29 20:51:56.024247176 +0000 UTC m=+266.273484696"
Jul 29 20:51:56.100789 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.100746    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.367838 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.367754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.472131 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.472097    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:56.474762 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.474739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:56.560036 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.560004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.660257 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:56.660175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:57.008818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.008788    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:57.010032 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.010013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5799b9b4f7-h2vnm"
Jul 29 20:51:57.164108 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.164060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:57.779111 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.779073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:57.877468 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.877436    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:57.977064 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:57.977037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:58.021832 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:58.021799    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl" event={"ID":"52d152d2-dcf1-46bd-a446-b0de2a577e7a","Type":"ContainerStarted","Data":"e9b1375f68802b7e93bc814b35aa11bd1069cba25a82eb2d41b0ad9017cdc887"}
Jul 29 20:51:58.035770 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:58.035742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:58.199423 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:58.199368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:58.367635 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:58.367546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:59.026397 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.026358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl" event={"ID":"52d152d2-dcf1-46bd-a446-b0de2a577e7a","Type":"ContainerStarted","Data":"76926f7671f22106e99e9761799c90ef2f2f172ed4dc5b7523bbf864c0ec8351"}
Jul 29 20:51:59.040797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.040752    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tdsbl" podStartSLOduration=18.059776053 podStartE2EDuration="20.040738299s" totalImagesPullingTime="1.980962246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:59.040260542 +0000 UTC m=+269.289498070" watchObservedRunningTime="2026-07-29 20:51:59.040738299 +0000 UTC m=+269.289975814"
Jul 29 20:51:59.167116 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.167075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:59.189820 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.189792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:59.189930 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.189835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:59.190454 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.190431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5787257-a1af-438f-80f8-b503a6463853-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:59.192225 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.192201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5787257-a1af-438f-80f8-b503a6463853-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p97db\" (UID: \"f5787257-a1af-438f-80f8-b503a6463853\") " pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:59.272611 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.272580    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p97db"
Jul 29 20:51:59.364910 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.364878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:59.399606 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.399579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p97db"]
Jul 29 20:51:59.402719 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:51:59.402689    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5787257_a1af_438f_80f8_b503a6463853.slice/crio-48588454d340e0dbe846d5d0e94862cfecd56b3506953a8aee5fccd79e432820 WatchSource:0}: Error finding container 48588454d340e0dbe846d5d0e94862cfecd56b3506953a8aee5fccd79e432820: Status 404 returned error can't find the container with id 48588454d340e0dbe846d5d0e94862cfecd56b3506953a8aee5fccd79e432820
Jul 29 20:51:59.566991 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:51:59.566895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:00.037723 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:00.037644    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p97db" event={"ID":"f5787257-a1af-438f-80f8-b503a6463853","Type":"ContainerStarted","Data":"48588454d340e0dbe846d5d0e94862cfecd56b3506953a8aee5fccd79e432820"}
Jul 29 20:52:00.831808 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:00.831767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:00.886463 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:00.886362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:03.257056 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:03.257027    2581 scope.go:117] "RemoveContainer" containerID="85b40c3b70b924c7f7f5b39f9fcaf38d7a4e98c952e1aa30f19a78b4f0360658"
Jul 29 20:52:05.063636 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.061284    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:52:05.063636 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.061461    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" event={"ID":"a8b0866d-ea8c-487d-9cec-9b85d83a185e","Type":"ContainerStarted","Data":"931fe34682089744740928d6c6698efd55e7f575039a624e4f51d02e9c0c2d6f"}
Jul 29 20:52:05.063636 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.062465    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:52:05.066755 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.066683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p97db" event={"ID":"f5787257-a1af-438f-80f8-b503a6463853","Type":"ContainerStarted","Data":"de43e7bb7fd0ec8c30f88ed9570a338162119acaaf7c16e77b5381ddf2a4e4fb"}
Jul 29 20:52:05.069456 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.069430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z"
Jul 29 20:52:05.098728 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.098501    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-9v96z" podStartSLOduration=23.382711435 podStartE2EDuration="26.09847884s" totalImagesPullingTime="2.715767405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:05.096862281 +0000 UTC m=+275.346099798" watchObservedRunningTime="2026-07-29 20:52:05.09847884 +0000 UTC m=+275.347716356"
Jul 29 20:52:05.117310 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.117275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.210527 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.210462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.262646 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.262608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.376810 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.376720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.752073 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.751945    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.804396 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.804331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:05.986691 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:05.986656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:06.047650 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:06.047565    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:07.006336 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:07.006296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:07.167872 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:07.167834    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:07.269592 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:07.269506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:08.080690 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:08.080655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p97db" event={"ID":"f5787257-a1af-438f-80f8-b503a6463853","Type":"ContainerStarted","Data":"f0df7ea92d23dc0a35331e1ec1ac4554e5dca38b0082e8c8bc5f56f9eca70c38"}
Jul 29 20:52:08.300296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:08.300265    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574cb46999-db2ts"]
Jul 29 20:52:08.300579 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:08.300551    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-574cb46999-db2ts" podUID="efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"
Jul 29 20:52:08.412965 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:08.412867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:08.510095 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:08.510054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:09.084360 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.084330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:52:09.091626 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.091600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:52:09.179288 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179260    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179288 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179292    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179572 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179572 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179419    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179572 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64zhn\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179714 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179573    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179714 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration\") pod \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\" (UID: \"efa1a824-7b5c-4a10-bfed-c59adbfdc2c8\") "
Jul 29 20:52:09.179714 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:52:09.179849 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179786    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:52:09.179934 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179918    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-ca-trust-extracted\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.179988 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.179940    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-certificates\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.180063 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.180039    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:52:09.182589 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.182548    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:52:09.182728 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.182592    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn" (OuterVolumeSpecName: "kube-api-access-64zhn") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "kube-api-access-64zhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:52:09.182728 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.182637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:52:09.182831 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.182808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" (UID: "efa1a824-7b5c-4a10-bfed-c59adbfdc2c8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:52:09.280929 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.280890    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64zhn\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-kube-api-access-64zhn\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.280929 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.280928    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-trusted-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.281147 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.280944    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-image-registry-private-configuration\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.281147 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.280958    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-bound-sa-token\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:09.281147 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:09.280976    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-installation-pull-secrets\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:10.090480 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:10.089348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574cb46999-db2ts"
Jul 29 20:52:10.124556 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:10.123495    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574cb46999-db2ts"]
Jul 29 20:52:10.143305 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:10.143267    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-574cb46999-db2ts"]
Jul 29 20:52:10.188756 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:10.188724    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8-registry-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:52:10.261511 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:10.261480    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa1a824-7b5c-4a10-bfed-c59adbfdc2c8" path="/var/lib/kubelet/pods/efa1a824-7b5c-4a10-bfed-c59adbfdc2c8/volumes"
Jul 29 20:52:12.098360 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:12.098323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p97db" event={"ID":"f5787257-a1af-438f-80f8-b503a6463853","Type":"ContainerStarted","Data":"28df73f2c94959be2b66563c9cdf135fe259436fec51ddfbc67902a461bc85a1"}
Jul 29 20:52:12.116401 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:12.116320    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p97db" podStartSLOduration=17.434542284 podStartE2EDuration="29.116304202s" totalImagesPullingTime="11.681761918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:51:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:12.115854261 +0000 UTC m=+282.365091777" watchObservedRunningTime="2026-07-29 20:52:12.116304202 +0000 UTC m=+282.365541716"
Jul 29 20:52:12.726997 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:12.726955    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2fn8c" podUID="80a2cb8b-db33-4b5a-84c7-8e18d332a1d5"
Jul 29 20:52:12.982984 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:12.982886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:13.020115 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.020086    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"]
Jul 29 20:52:13.023252 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.023231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.025541 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.025521    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 20:52:13.025832 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.025817    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 20:52:13.026350 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.026336    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 20:52:13.026885 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.026864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 20:52:13.026985 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.026882    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 20:52:13.026985 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.026866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h6xjs\""
Jul 29 20:52:13.031193 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.031166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"]
Jul 29 20:52:13.050222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.050196    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:13.102017 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.101984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:13.106106 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.106083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:13.111703 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.111679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.111812 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.111796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.111889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.111850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrjmp\" (UniqueName: \"kubernetes.io/projected/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-kube-api-access-jrjmp\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.111944 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.111883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.213054 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.213010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.213054 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.213059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrjmp\" (UniqueName: \"kubernetes.io/projected/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-kube-api-access-jrjmp\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.213299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.213080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.213299 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.213117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.217898 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.214260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.219080 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.219059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.220143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.220117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.221398 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.221366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrjmp\" (UniqueName: \"kubernetes.io/projected/2e82cc2d-10cf-4fdc-84bf-92916e7b58a3-kube-api-access-jrjmp\") pod \"prometheus-operator-84d69f794d-c4vvj\" (UID: \"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.333724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.333683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"
Jul 29 20:52:13.460830 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:13.460799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-c4vvj"]
Jul 29 20:52:13.463692 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:13.463655    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e82cc2d_10cf_4fdc_84bf_92916e7b58a3.slice/crio-2374306f47556354ed9e480c0c3f5bec3487b6db7c0311046dd1f824f4a530ba WatchSource:0}: Error finding container 2374306f47556354ed9e480c0c3f5bec3487b6db7c0311046dd1f824f4a530ba: Status 404 returned error can't find the container with id 2374306f47556354ed9e480c0c3f5bec3487b6db7c0311046dd1f824f4a530ba
Jul 29 20:52:14.107018 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:14.106984    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj" event={"ID":"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3","Type":"ContainerStarted","Data":"2374306f47556354ed9e480c0c3f5bec3487b6db7c0311046dd1f824f4a530ba"}
Jul 29 20:52:14.972528 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:14.972493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:15.112483 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:15.112443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj" event={"ID":"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3","Type":"ContainerStarted","Data":"1129008a4659299dd9771164ffa54dacfdd2ecfb21f326bfd4730f1a20fc74ee"}
Jul 29 20:52:15.112931 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:15.112494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj" event={"ID":"2e82cc2d-10cf-4fdc-84bf-92916e7b58a3","Type":"ContainerStarted","Data":"f81af98c4d8d28f9f1f447e9ed2def745edd8114d7c9a50fe7087204f648bdeb"}
Jul 29 20:52:15.130092 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:15.130042    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-c4vvj" podStartSLOduration=0.866177449 podStartE2EDuration="2.130025817s" totalImagesPullingTime="1.263848368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:15.129020486 +0000 UTC m=+285.378258003" watchObservedRunningTime="2026-07-29 20:52:15.130025817 +0000 UTC m=+285.379263332"
Jul 29 20:52:16.139784 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:16.139750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:16.142240 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:16.142217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80a2cb8b-db33-4b5a-84c7-8e18d332a1d5-metrics-tls\") pod \"dns-default-2fn8c\" (UID: \"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5\") " pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:16.405209 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:16.405125    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qgjxr\""
Jul 29 20:52:16.413856 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:16.413836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:16.543496 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:16.543466    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2fn8c"]
Jul 29 20:52:16.546917 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:16.546883    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a2cb8b_db33_4b5a_84c7_8e18d332a1d5.slice/crio-17547d0cf0c4752920bc570a25911fe88a4e67c6d02d9052578d560a82ee7e3f WatchSource:0}: Error finding container 17547d0cf0c4752920bc570a25911fe88a4e67c6d02d9052578d560a82ee7e3f: Status 404 returned error can't find the container with id 17547d0cf0c4752920bc570a25911fe88a4e67c6d02d9052578d560a82ee7e3f
Jul 29 20:52:17.121206 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.121169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fn8c" event={"ID":"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5","Type":"ContainerStarted","Data":"17547d0cf0c4752920bc570a25911fe88a4e67c6d02d9052578d560a82ee7e3f"}
Jul 29 20:52:17.138232 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.138097    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.257319 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.257285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.359176 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.359138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.384082 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.383994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"]
Jul 29 20:52:17.386891 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.386732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.389634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.389088    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2slcw\""
Jul 29 20:52:17.389634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.389293    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 20:52:17.389634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.389530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 20:52:17.403469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.403430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"]
Jul 29 20:52:17.404185 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.404163    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tfcd7"]
Jul 29 20:52:17.406891 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.406869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.410505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.409865    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 20:52:17.410505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.409928    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 20:52:17.410505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.410137    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 20:52:17.410505 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.410333    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5q2kp\""
Jul 29 20:52:17.417561 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.417489    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"]
Jul 29 20:52:17.420870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.420854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.429402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.424634    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 20:52:17.429402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.424651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7q8qw\""
Jul 29 20:52:17.429402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.424830    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 20:52:17.429402 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.425052    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 20:52:17.433204 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.432970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"]
Jul 29 20:52:17.454247 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.454210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.454568 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.454541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.454684 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.454667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17496c2d-27e8-4d9d-9b38-66b1d391c818-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.454763 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.454718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcxth\" (UniqueName: \"kubernetes.io/projected/17496c2d-27e8-4d9d-9b38-66b1d391c818-kube-api-access-qcxth\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.464125 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.463893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.535398 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.534961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.555707 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.555647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-textfile\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.555921 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.555722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-wtmp\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.555921 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.555804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.555921 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.555883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-root\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.556168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.555943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.556168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17496c2d-27e8-4d9d-9b38-66b1d391c818-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.556168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-sys\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.556168 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wljrj\" (UniqueName: \"kubernetes.io/projected/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-api-access-wljrj\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.556371 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcxth\" (UniqueName: \"kubernetes.io/projected/17496c2d-27e8-4d9d-9b38-66b1d391c818-kube-api-access-qcxth\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.556371 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.556944 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-metrics-client-ca\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.556944 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17496c2d-27e8-4d9d-9b38-66b1d391c818-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.557099 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.556940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.557099 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.557099 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.557099 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.557134    2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744g8\" (UniqueName: \"kubernetes.io/projected/b0623358-ed78-467a-a8c8-f394678da67f-kube-api-access-744g8\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.557212    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls podName:17496c2d-27e8-4d9d-9b38-66b1d391c818 nodeName:}" failed. No retries permitted until 2026-07-29 20:52:18.057192064 +0000 UTC m=+288.306429563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-hjr57" (UID: "17496c2d-27e8-4d9d-9b38-66b1d391c818") : secret "openshift-state-metrics-tls" not found
Jul 29 20:52:17.557296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.557232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.561279 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.561255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.568110 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.568065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcxth\" (UniqueName: \"kubernetes.io/projected/17496c2d-27e8-4d9d-9b38-66b1d391c818-kube-api-access-qcxth\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:17.653434 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.653251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.658067 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-metrics-client-ca\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744g8\" (UniqueName: \"kubernetes.io/projected/b0623358-ed78-467a-a8c8-f394678da67f-kube-api-access-744g8\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.658277 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-textfile\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-wtmp\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-root\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-sys\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wljrj\" (UniqueName: \"kubernetes.io/projected/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-api-access-wljrj\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.658657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.658531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.659126 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.659102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-textfile\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.659292 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.659222    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 20:52:17.659292 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.659287    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls podName:b0623358-ed78-467a-a8c8-f394678da67f nodeName:}" failed. No retries permitted until 2026-07-29 20:52:18.159269315 +0000 UTC m=+288.408506814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls") pod "node-exporter-tfcd7" (UID: "b0623358-ed78-467a-a8c8-f394678da67f") : secret "node-exporter-tls" not found
Jul 29 20:52:17.660043 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.660000    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 20:52:17.660154 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:17.660068    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls podName:5686bbdc-efb4-4c49-8cd1-1eb9b053e429 nodeName:}" failed. No retries permitted until 2026-07-29 20:52:18.160052941 +0000 UTC m=+288.409290438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-45c7j" (UID: "5686bbdc-efb4-4c49-8cd1-1eb9b053e429") : secret "kube-state-metrics-tls" not found
Jul 29 20:52:17.660154 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.660115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-root\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.660267 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.660236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-wtmp\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.660696 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.660674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.660781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.660736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0623358-ed78-467a-a8c8-f394678da67f-sys\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.661369 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.661343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.661480 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.661454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0623358-ed78-467a-a8c8-f394678da67f-metrics-client-ca\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.662962 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.662936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.663682 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.663641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.665018 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.664971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.667222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.667172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.683048 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.682978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-744g8\" (UniqueName: \"kubernetes.io/projected/b0623358-ed78-467a-a8c8-f394678da67f-kube-api-access-744g8\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:17.685075 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.684990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wljrj\" (UniqueName: \"kubernetes.io/projected/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-api-access-wljrj\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:17.778826 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.778723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.878977 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.878939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:17.978104 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:17.978002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.063015 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.062981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:18.065778 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.065753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17496c2d-27e8-4d9d-9b38-66b1d391c818-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hjr57\" (UID: \"17496c2d-27e8-4d9d-9b38-66b1d391c818\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:18.078159 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.078125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.163807 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.163745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:18.163992 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.163826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:18.166370 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.166345    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0623358-ed78-467a-a8c8-f394678da67f-node-exporter-tls\") pod \"node-exporter-tfcd7\" (UID: \"b0623358-ed78-467a-a8c8-f394678da67f\") " pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:18.166510 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.166488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5686bbdc-efb4-4c49-8cd1-1eb9b053e429-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-45c7j\" (UID: \"5686bbdc-efb4-4c49-8cd1-1eb9b053e429\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:18.178729 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.178690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.302217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.302186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"
Jul 29 20:52:18.323732 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.323700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfcd7"
Jul 29 20:52:18.336525 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:18.336497    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0623358_ed78_467a_a8c8_f394678da67f.slice/crio-7cd9c716fce278ba621de4ae9d1df214996523d5dc754ee2f56223f15ad82073 WatchSource:0}: Error finding container 7cd9c716fce278ba621de4ae9d1df214996523d5dc754ee2f56223f15ad82073: Status 404 returned error can't find the container with id 7cd9c716fce278ba621de4ae9d1df214996523d5dc754ee2f56223f15ad82073
Jul 29 20:52:18.343341 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.343316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"
Jul 29 20:52:18.372260 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.372072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.457738 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.457698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.463942 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.461763    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:52:18.472104 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.468819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473145    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473481    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473667    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473729    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473748    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473818    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473891    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 20:52:18.474020 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.473938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vnjsd\""
Jul 29 20:52:18.484498 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.481536    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:52:18.506805 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.505305    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57"]
Jul 29 20:52:18.530462 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.530432    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j"]
Jul 29 20:52:18.534137 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:18.534109    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5686bbdc_efb4_4c49_8cd1_1eb9b053e429.slice/crio-7c3c624812638fd697d2849b0743d83c9d536d1e01bb9528d873dd47bf02f2ca WatchSource:0}: Error finding container 7c3c624812638fd697d2849b0743d83c9d536d1e01bb9528d873dd47bf02f2ca: Status 404 returned error can't find the container with id 7c3c624812638fd697d2849b0743d83c9d536d1e01bb9528d873dd47bf02f2ca
Jul 29 20:52:18.571801 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.571939 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.571939 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.571939 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.571939 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571891    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.571976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572293 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9r8\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572293 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572293 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.572542 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.572293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.584620 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.584591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:18.673589 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.673767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.673767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.673767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.673767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9r8\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.673767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.673986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.674008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674039 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.674033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.674405 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.674356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.675372 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:18.675346    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle podName:a9bbf1f2-54dc-416d-841b-0895d96adeba nodeName:}" failed. No retries permitted until 2026-07-29 20:52:19.175321039 +0000 UTC m=+289.424558549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba") : configmap references non-existent config key: ca-bundle.crt
Jul 29 20:52:18.676404 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.676364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.677947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.678760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.678854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.679108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680217 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.680074    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.680577 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.680515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.681073 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.681040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.682128 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.682099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.682610 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.682590    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:18.685758 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:18.685737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9r8\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:19.132809 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.132772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfcd7" event={"ID":"b0623358-ed78-467a-a8c8-f394678da67f","Type":"ContainerStarted","Data":"7cd9c716fce278ba621de4ae9d1df214996523d5dc754ee2f56223f15ad82073"}
Jul 29 20:52:19.134877 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.134844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fn8c" event={"ID":"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5","Type":"ContainerStarted","Data":"471e8e56442be7abed60b892d741a8e804bd94d4376c17dd5bb94ae880044fe7"}
Jul 29 20:52:19.134877 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.134904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2fn8c" event={"ID":"80a2cb8b-db33-4b5a-84c7-8e18d332a1d5","Type":"ContainerStarted","Data":"11083cca0566247f128d09782e83e56fa9043150bd5b50e5e3190cc45a86f97e"}
Jul 29 20:52:19.135099 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.135013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:19.137314 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.137285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57" event={"ID":"17496c2d-27e8-4d9d-9b38-66b1d391c818","Type":"ContainerStarted","Data":"b5b0f7537f80e982278af016f738948ea0bd1a3e21cd31607288ae81a298f624"}
Jul 29 20:52:19.137472 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.137349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57" event={"ID":"17496c2d-27e8-4d9d-9b38-66b1d391c818","Type":"ContainerStarted","Data":"d037f4dd05ad1ff4ae196aadaf540536be6c61cf029015cf2ef066913013857e"}
Jul 29 20:52:19.137472 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.137409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57" event={"ID":"17496c2d-27e8-4d9d-9b38-66b1d391c818","Type":"ContainerStarted","Data":"c76f9aafe4c261025598c9314ed0d06056c16b33d1d10d93323c08bdd7bf4078"}
Jul 29 20:52:19.138643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.138623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j" event={"ID":"5686bbdc-efb4-4c49-8cd1-1eb9b053e429","Type":"ContainerStarted","Data":"7c3c624812638fd697d2849b0743d83c9d536d1e01bb9528d873dd47bf02f2ca"}
Jul 29 20:52:19.152129 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.152080    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2fn8c" podStartSLOduration=251.369707771 podStartE2EDuration="4m13.152062963s" totalImagesPullingTime="1.782355192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:19.151074249 +0000 UTC m=+289.400311767" watchObservedRunningTime="2026-07-29 20:52:19.152062963 +0000 UTC m=+289.401300523"
Jul 29 20:52:19.179550 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.179521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:19.180754 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.180730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:19.387767 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.387678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:52:19.579795 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:19.579763    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:52:19.761204 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:19.761166    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bbf1f2_54dc_416d_841b_0895d96adeba.slice/crio-ddfe1a4833f37cf1dbafd952a095432f39d9a555b66563564eebc00498a7d032 WatchSource:0}: Error finding container ddfe1a4833f37cf1dbafd952a095432f39d9a555b66563564eebc00498a7d032: Status 404 returned error can't find the container with id ddfe1a4833f37cf1dbafd952a095432f39d9a555b66563564eebc00498a7d032
Jul 29 20:52:20.146027 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.145996    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0623358-ed78-467a-a8c8-f394678da67f" containerID="849d0dfb0b590e1aa69b1831080cf9c75e335725528700fb71d5ca066d3773f0" exitCode=0
Jul 29 20:52:20.146151 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.146073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfcd7" event={"ID":"b0623358-ed78-467a-a8c8-f394678da67f","Type":"ContainerDied","Data":"849d0dfb0b590e1aa69b1831080cf9c75e335725528700fb71d5ca066d3773f0"}
Jul 29 20:52:20.153755 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.153725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"ddfe1a4833f37cf1dbafd952a095432f39d9a555b66563564eebc00498a7d032"}
Jul 29 20:52:20.322535 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.321756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:20.418250 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.418140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:20.475353 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.475317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:20.584205 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.584155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:20.787531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:20.787494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:21.159870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.159836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfcd7" event={"ID":"b0623358-ed78-467a-a8c8-f394678da67f","Type":"ContainerStarted","Data":"51b6514f1d065dba19923097b78c78509ce0836a18d19062febe120701d7fc5a"}
Jul 29 20:52:21.160000 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.159889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfcd7" event={"ID":"b0623358-ed78-467a-a8c8-f394678da67f","Type":"ContainerStarted","Data":"ecb30050540a2e5fcda1cf0ec160afab783965c2367501ee89a6d5c606dc2f0f"}
Jul 29 20:52:21.163760 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.161879    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955" exitCode=0
Jul 29 20:52:21.163760 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.161990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"}
Jul 29 20:52:21.165795 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.165761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57" event={"ID":"17496c2d-27e8-4d9d-9b38-66b1d391c818","Type":"ContainerStarted","Data":"ddb1561da91a4bb300764356ae3bd0461e12bb34ab40c5e1c8de0f5a2425b300"}
Jul 29 20:52:21.168206 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.168173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j" event={"ID":"5686bbdc-efb4-4c49-8cd1-1eb9b053e429","Type":"ContainerStarted","Data":"6281ebfbed94467268bc110b83d057fe293d317fed74f7d8ecd24e3de6821ef9"}
Jul 29 20:52:21.168304 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.168219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j" event={"ID":"5686bbdc-efb4-4c49-8cd1-1eb9b053e429","Type":"ContainerStarted","Data":"56b0fc92ce5461dd22f287fa488414f18d7c7ef842c68dbad22a8de1e38e270d"}
Jul 29 20:52:21.168304 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.168239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j" event={"ID":"5686bbdc-efb4-4c49-8cd1-1eb9b053e429","Type":"ContainerStarted","Data":"f863a4b6b892a72d1710095d821a1d61a53785f32e98fff1a1ddf79d6022ade6"}
Jul 29 20:52:21.183666 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.183584    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tfcd7" podStartSLOduration=3.322361803 podStartE2EDuration="4.183566061s" totalImagesPullingTime="783.208845ms" totalInitContainerRuntime="77.995413ms" isStatefulPod=true podCreationTimestamp="2026-07-29 20:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:21.182933876 +0000 UTC m=+291.432171393" watchObservedRunningTime="2026-07-29 20:52:21.183566061 +0000 UTC m=+291.432803560"
Jul 29 20:52:21.226851 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.226792    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hjr57" podStartSLOduration=2.754286409 podStartE2EDuration="4.226776492s" totalImagesPullingTime="1.472490083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:21.226503126 +0000 UTC m=+291.475740647" watchObservedRunningTime="2026-07-29 20:52:21.226776492 +0000 UTC m=+291.476014007"
Jul 29 20:52:21.245212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:21.245159    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-45c7j" podStartSLOduration=2.665633415 podStartE2EDuration="4.245143569s" totalImagesPullingTime="1.579510154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:21.243963474 +0000 UTC m=+291.493201012" watchObservedRunningTime="2026-07-29 20:52:21.245143569 +0000 UTC m=+291.494381083"
Jul 29 20:52:22.158863 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.158819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:22.290802 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.290735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:22.457197 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.457107    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:22.581003 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.580975    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"]
Jul 29 20:52:22.585181 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.585158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.587694 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.587670    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 20:52:22.587815 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.587670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 20:52:22.587815 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.587794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 20:52:22.587934 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.587816    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 20:52:22.588139 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.588120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 20:52:22.588222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.588180    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4lwx7\""
Jul 29 20:52:22.592199 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.592178    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 20:52:22.597333 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.597308    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"]
Jul 29 20:52:22.716011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.715978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-serving-certs-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-metrics-client-ca\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716248 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716248 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-federate-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716307 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.716339 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.716305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5dcw\" (UniqueName: \"kubernetes.io/projected/67501f0d-06ad-4e50-a743-b8e114129050-kube-api-access-j5dcw\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817199 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-metrics-client-ca\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817331 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817331 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-federate-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817331 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817331 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5dcw\" (UniqueName: \"kubernetes.io/projected/67501f0d-06ad-4e50-a743-b8e114129050-kube-api-access-j5dcw\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-serving-certs-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.817578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.817447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.818095 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.818070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-metrics-client-ca\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.818178 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.818106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-serving-certs-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.818233 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.818216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.820155 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.820135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-federate-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.820458 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.820440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.820530 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.820502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-secret-telemeter-client\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.820568 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.820498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67501f0d-06ad-4e50-a743-b8e114129050-telemeter-client-tls\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.825620 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.825597    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5dcw\" (UniqueName: \"kubernetes.io/projected/67501f0d-06ad-4e50-a743-b8e114129050-kube-api-access-j5dcw\") pod \"telemeter-client-7c887d67c5-f8qgc\" (UID: \"67501f0d-06ad-4e50-a743-b8e114129050\") " pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:22.914529 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:22.914498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"
Jul 29 20:52:23.055307 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.055273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c887d67c5-f8qgc"]
Jul 29 20:52:23.058570 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:23.058545    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67501f0d_06ad_4e50_a743_b8e114129050.slice/crio-d5c5f55ee17e7bc29fa2c13d32f838234ddac3d60a959c599c9ce2085146d993 WatchSource:0}: Error finding container d5c5f55ee17e7bc29fa2c13d32f838234ddac3d60a959c599c9ce2085146d993: Status 404 returned error can't find the container with id d5c5f55ee17e7bc29fa2c13d32f838234ddac3d60a959c599c9ce2085146d993
Jul 29 20:52:23.179319 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.179287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"}
Jul 29 20:52:23.179743 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.179361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"}
Jul 29 20:52:23.179743 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.179419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"}
Jul 29 20:52:23.179743 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.179452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"}
Jul 29 20:52:23.179743 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.179475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"}
Jul 29 20:52:23.180377 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.180357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc" event={"ID":"67501f0d-06ad-4e50-a743-b8e114129050","Type":"ContainerStarted","Data":"d5c5f55ee17e7bc29fa2c13d32f838234ddac3d60a959c599c9ce2085146d993"}
Jul 29 20:52:23.599603 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.599569    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:52:23.604233 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.604194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.608262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.608223    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 20:52:23.608262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.608238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 20:52:23.608262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.608247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 20:52:23.608534 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.608356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 20:52:23.609221 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609150    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cs2s5\""
Jul 29 20:52:23.609221 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 20:52:23.609414 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609228    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5rafhlac0ss2v\""
Jul 29 20:52:23.609414 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 20:52:23.609536 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609432    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 20:52:23.609668 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.609645    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 20:52:23.610141 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.610111    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 20:52:23.611134 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.610998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:23.611881 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.611857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 20:52:23.612511 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.612492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 20:52:23.612974 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.612957    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 20:52:23.614255 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.614231    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 20:52:23.614676 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.614233    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 20:52:23.619855 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.619832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 20:52:23.624657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.624634    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:52:23.729979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.729941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.729979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.729979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730010    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730171 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.730614 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.730601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtq7d\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.831842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.831842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.831842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.831842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.831985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832144 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtq7d\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.832578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.832295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.833088 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.833063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.833662 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.833638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.838633 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.834720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.838633 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.836350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.838633 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.837070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.838633 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.837429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.838633 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.838418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.841041 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.839672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.841041 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.840197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.841041 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.840240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.841041 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.840938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.843675 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.843628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.844870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.844224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.844870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.844270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.844870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.844696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.844870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.844799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.844870 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.844831    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.845187 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.845033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.846050 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.846010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.848964 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.848917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtq7d\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d\") pod \"prometheus-k8s-0\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:23.927082 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:23.926973    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:24.117584 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:24.117542    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:52:24.122191 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:52:24.122155    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241f12bf_fee9_4d97_8d8c_ac245499282c.slice/crio-45c3bebe67165c4b0bf30a4dc401b583e06897667e92c473b6e27e70aeb2e583 WatchSource:0}: Error finding container 45c3bebe67165c4b0bf30a4dc401b583e06897667e92c473b6e27e70aeb2e583: Status 404 returned error can't find the container with id 45c3bebe67165c4b0bf30a4dc401b583e06897667e92c473b6e27e70aeb2e583
Jul 29 20:52:24.187452 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:24.187417    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"45c3bebe67165c4b0bf30a4dc401b583e06897667e92c473b6e27e70aeb2e583"}
Jul 29 20:52:24.191061 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:24.191024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerStarted","Data":"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"}
Jul 29 20:52:24.218542 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:24.218480    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.783581089 podStartE2EDuration="6.21845834s" totalImagesPullingTime="3.415232939s" totalInitContainerRuntime="19.644312ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:24.216567981 +0000 UTC m=+294.465805513" watchObservedRunningTime="2026-07-29 20:52:24.21845834 +0000 UTC m=+294.467695878"
Jul 29 20:52:25.198462 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:25.198428    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7" exitCode=0
Jul 29 20:52:25.198889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:25.198574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"}
Jul 29 20:52:26.205992 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:26.205949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc" event={"ID":"67501f0d-06ad-4e50-a743-b8e114129050","Type":"ContainerStarted","Data":"9bdcd23895ca9642a1da9652a88ddc88e621fd69c0934e26b27d1349799a18cf"}
Jul 29 20:52:26.206425 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:26.206068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc" event={"ID":"67501f0d-06ad-4e50-a743-b8e114129050","Type":"ContainerStarted","Data":"5f60d21b93a80abe81f9bee34953b90de212d102d370a3ad06f247d36d1d431f"}
Jul 29 20:52:26.206425 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:26.206102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc" event={"ID":"67501f0d-06ad-4e50-a743-b8e114129050","Type":"ContainerStarted","Data":"831fbbec92de1022ed39e9b50789a7a4a88a6003a984021b53f5ff37220f2c91"}
Jul 29 20:52:26.227574 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:26.227517    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c887d67c5-f8qgc" podStartSLOduration=2.01937241 podStartE2EDuration="4.227500464s" totalImagesPullingTime="2.208128054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:26.225648265 +0000 UTC m=+296.474885794" watchObservedRunningTime="2026-07-29 20:52:26.227500464 +0000 UTC m=+296.476737978"
Jul 29 20:52:26.257738 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:52:26.257699    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ldb4s" podUID="8b7f12a9-0f65-492a-8764-a3d0ace547fe"
Jul 29 20:52:27.111159 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:27.111120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:27.211951 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:27.211903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:29.156585 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:29.156545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2fn8c"
Jul 29 20:52:29.221794 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:29.221761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"}
Jul 29 20:52:29.221955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:29.221838    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"}
Jul 29 20:52:29.583878 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:29.583837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:29.684888 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:29.684850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:30.341375 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:30.341171    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:30.442430 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:30.442370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:30.538842 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:30.538811    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:52:30.539797 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:30.539769    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:52:30.549174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:30.549156    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 20:52:31.232869 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.232832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"}
Jul 29 20:52:31.233033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.232926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"}
Jul 29 20:52:31.233033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.232951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"}
Jul 29 20:52:31.233033 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.232988    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerStarted","Data":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"}
Jul 29 20:52:31.265959 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.265909    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.060360291 podStartE2EDuration="8.265892231s" totalImagesPullingTime="5.185519299s" totalInitContainerRuntime="20.012641ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:52:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:52:31.264619795 +0000 UTC m=+301.513857353" watchObservedRunningTime="2026-07-29 20:52:31.265892231 +0000 UTC m=+301.515129747"
Jul 29 20:52:31.613175 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.613131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:31.713495 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:31.713455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:33.927697 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:33.927659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:52:34.658571 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:34.658534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:35.896636 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:35.896592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:36.763391 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:36.763356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:37.257312 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:37.257287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:52:37.371647 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:37.371615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:37.473375 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:37.473339    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:37.891334 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:37.891294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:37.940524 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:37.940483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:38.263618 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:38.263583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:38.319076 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:38.319038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:41.289958 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:41.289926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:41.343348 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:41.343316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:41.690236 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:41.690146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:41.745988 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:41.745954    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:41.799723 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:41.799687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:45.834535 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:45.834502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:45.937125 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:45.937093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:50.365914 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:50.365869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:50.420977 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:52:50.420939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:08.390359 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:08.390323    2581 generic.go:356] "Generic (PLEG): container finished" podID="12a802bb-5da0-4c97-b698-997813f976a3" containerID="affde5860208d6245cf9496bd0cda059a568bbc25b13e4bf62b4fe3377e7b0ff" exitCode=0
Jul 29 20:53:08.390803 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:08.390418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7" event={"ID":"12a802bb-5da0-4c97-b698-997813f976a3","Type":"ContainerDied","Data":"affde5860208d6245cf9496bd0cda059a568bbc25b13e4bf62b4fe3377e7b0ff"}
Jul 29 20:53:08.390853 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:08.390841    2581 scope.go:117] "RemoveContainer" containerID="affde5860208d6245cf9496bd0cda059a568bbc25b13e4bf62b4fe3377e7b0ff"
Jul 29 20:53:08.835492 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:08.835453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:08.935571 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:08.935491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:09.396578 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:09.396542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2nmb7" event={"ID":"12a802bb-5da0-4c97-b698-997813f976a3","Type":"ContainerStarted","Data":"21917ece21fad5a879d406e89ce6e65ab98a6801b8cd993032a39dcb87420e7c"}
Jul 29 20:53:15.827987 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:15.827952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:15.891667 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:15.891633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:19.921006 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:19.920968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:19.977795 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:19.977753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:23.927745 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:23.927650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:23.944047 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:23.944022    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:24.474242 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:24.474215    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:33.461644 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:33.461610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:33.520139 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:33.520107    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:36.824067 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:36.824032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:36.879780 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:36.879749    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:37.618486 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.618446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:37.721092 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.721053    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:37.722052 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:37.722405 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722345    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="alertmanager" containerID="cri-o://cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5" gracePeriod=120
Jul 29 20:53:37.722531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722370    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy" containerID="cri-o://e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532" gracePeriod=120
Jul 29 20:53:37.722531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722435    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="config-reloader" containerID="cri-o://2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef" gracePeriod=120
Jul 29 20:53:37.722531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722416    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-web" containerID="cri-o://6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e" gracePeriod=120
Jul 29 20:53:37.722531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722438    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-metric" containerID="cri-o://c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00" gracePeriod=120
Jul 29 20:53:37.722531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:37.722407    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="prom-label-proxy" containerID="cri-o://9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a" gracePeriod=120
Jul 29 20:53:38.515096 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515056    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a" exitCode=0
Jul 29 20:53:38.515096 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515084    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532" exitCode=0
Jul 29 20:53:38.515096 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515090    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef" exitCode=0
Jul 29 20:53:38.515096 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515097    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5" exitCode=0
Jul 29 20:53:38.515595 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"}
Jul 29 20:53:38.515595 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"}
Jul 29 20:53:38.515595 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"}
Jul 29 20:53:38.515595 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.515249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"}
Jul 29 20:53:38.970198 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:38.970175    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.142920 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142834    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.142920 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142867    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.142920 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142887    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.142920 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.142971    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143004    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143095    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg9r8\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143144    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143194    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web\") pod \"a9bbf1f2-54dc-416d-841b-0895d96adeba\" (UID: \"a9bbf1f2-54dc-416d-841b-0895d96adeba\") "
Jul 29 20:53:39.143688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:39.143688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143308    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:39.143688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:53:39.143688 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143675    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-metrics-client-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.144106 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.143719    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.146240 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.146210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume" (OuterVolumeSpecName: "config-volume") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.146369 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.146289    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:53:39.147146 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.147112    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out" (OuterVolumeSpecName: "config-out") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:53:39.147146 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.147119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.147449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.147422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.147449 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.147442    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8" (OuterVolumeSpecName: "kube-api-access-mg9r8") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "kube-api-access-mg9r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:53:39.147586 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.147444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.148310 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.148295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.151209 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.151186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.158976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.158954    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config" (OuterVolumeSpecName: "web-config") pod "a9bbf1f2-54dc-416d-841b-0895d96adeba" (UID: "a9bbf1f2-54dc-416d-841b-0895d96adeba"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:39.244701 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244657    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-main-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244701 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244695    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244701 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244706    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244717    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-tls-assets\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244725    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244734    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-cluster-tls-config\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244743    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-web-config\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244753    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9bbf1f2-54dc-416d-841b-0895d96adeba-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244761    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-config-out\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244769    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9bbf1f2-54dc-416d-841b-0895d96adeba-alertmanager-main-db\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.244924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.244777    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg9r8\" (UniqueName: \"kubernetes.io/projected/a9bbf1f2-54dc-416d-841b-0895d96adeba-kube-api-access-mg9r8\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:39.522011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.521981    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00" exitCode=0
Jul 29 20:53:39.522011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522006    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerID="6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e" exitCode=0
Jul 29 20:53:39.522484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"}
Jul 29 20:53:39.522484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522062    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.522484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"}
Jul 29 20:53:39.522484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9bbf1f2-54dc-416d-841b-0895d96adeba","Type":"ContainerDied","Data":"ddfe1a4833f37cf1dbafd952a095432f39d9a555b66563564eebc00498a7d032"}
Jul 29 20:53:39.522484 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.522150    2581 scope.go:117] "RemoveContainer" containerID="9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"
Jul 29 20:53:39.531149 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.531102    2581 scope.go:117] "RemoveContainer" containerID="c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"
Jul 29 20:53:39.538826 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.538805    2581 scope.go:117] "RemoveContainer" containerID="e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"
Jul 29 20:53:39.545816 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.545795    2581 scope.go:117] "RemoveContainer" containerID="6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"
Jul 29 20:53:39.549146 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.549122    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:39.553236 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.553213    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:39.553816 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.553796    2581 scope.go:117] "RemoveContainer" containerID="2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"
Jul 29 20:53:39.561245 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.561228    2581 scope.go:117] "RemoveContainer" containerID="cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"
Jul 29 20:53:39.568627 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.568604    2581 scope.go:117] "RemoveContainer" containerID="39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"
Jul 29 20:53:39.576752 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.576728    2581 scope.go:117] "RemoveContainer" containerID="9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"
Jul 29 20:53:39.577057 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.577037    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a\": container with ID starting with 9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a not found: ID does not exist" containerID="9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"
Jul 29 20:53:39.577109 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577067    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"} err="failed to get container status \"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a\": rpc error: code = NotFound desc = could not find container \"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a\": container with ID starting with 9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a not found: ID does not exist"
Jul 29 20:53:39.577109 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577100    2581 scope.go:117] "RemoveContainer" containerID="c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"
Jul 29 20:53:39.577340 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.577326    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00\": container with ID starting with c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00 not found: ID does not exist" containerID="c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"
Jul 29 20:53:39.577406 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577344    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"} err="failed to get container status \"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00\": rpc error: code = NotFound desc = could not find container \"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00\": container with ID starting with c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00 not found: ID does not exist"
Jul 29 20:53:39.577406 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577358    2581 scope.go:117] "RemoveContainer" containerID="e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"
Jul 29 20:53:39.577645 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.577627    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532\": container with ID starting with e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532 not found: ID does not exist" containerID="e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"
Jul 29 20:53:39.577704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577652    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"} err="failed to get container status \"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532\": rpc error: code = NotFound desc = could not find container \"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532\": container with ID starting with e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532 not found: ID does not exist"
Jul 29 20:53:39.577704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577670    2581 scope.go:117] "RemoveContainer" containerID="6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"
Jul 29 20:53:39.577916 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.577901    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e\": container with ID starting with 6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e not found: ID does not exist" containerID="6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"
Jul 29 20:53:39.577953 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577924    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"} err="failed to get container status \"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e\": rpc error: code = NotFound desc = could not find container \"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e\": container with ID starting with 6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e not found: ID does not exist"
Jul 29 20:53:39.577953 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.577943    2581 scope.go:117] "RemoveContainer" containerID="2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"
Jul 29 20:53:39.578190 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.578176    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef\": container with ID starting with 2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef not found: ID does not exist" containerID="2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"
Jul 29 20:53:39.578230 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578194    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"} err="failed to get container status \"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef\": rpc error: code = NotFound desc = could not find container \"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef\": container with ID starting with 2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef not found: ID does not exist"
Jul 29 20:53:39.578230 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578206    2581 scope.go:117] "RemoveContainer" containerID="cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"
Jul 29 20:53:39.578556 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.578536    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5\": container with ID starting with cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5 not found: ID does not exist" containerID="cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"
Jul 29 20:53:39.578637 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578562    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"} err="failed to get container status \"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5\": rpc error: code = NotFound desc = could not find container \"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5\": container with ID starting with cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5 not found: ID does not exist"
Jul 29 20:53:39.578637 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578581    2581 scope.go:117] "RemoveContainer" containerID="39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"
Jul 29 20:53:39.578792 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:39.578775    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955\": container with ID starting with 39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955 not found: ID does not exist" containerID="39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"
Jul 29 20:53:39.578855 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578801    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"} err="failed to get container status \"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955\": rpc error: code = NotFound desc = could not find container \"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955\": container with ID starting with 39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955 not found: ID does not exist"
Jul 29 20:53:39.578855 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.578820    2581 scope.go:117] "RemoveContainer" containerID="9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"
Jul 29 20:53:39.579059 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579041    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a"} err="failed to get container status \"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a\": rpc error: code = NotFound desc = could not find container \"9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a\": container with ID starting with 9b9a2cab732f7f97974b4b3e1cffad2bd264236c03c5bdd09ac3d66d0b63613a not found: ID does not exist"
Jul 29 20:53:39.579114 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579059    2581 scope.go:117] "RemoveContainer" containerID="c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"
Jul 29 20:53:39.579293 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579274    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00"} err="failed to get container status \"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00\": rpc error: code = NotFound desc = could not find container \"c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00\": container with ID starting with c517e46b5fda4324b9efb5a84d4c6278a8e57a3b9e1bbedaa84c3d4d7617fe00 not found: ID does not exist"
Jul 29 20:53:39.579353 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579294    2581 scope.go:117] "RemoveContainer" containerID="e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"
Jul 29 20:53:39.579557 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579536    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532"} err="failed to get container status \"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532\": rpc error: code = NotFound desc = could not find container \"e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532\": container with ID starting with e8ec597c7e81a764d84e76bedc5e31a38395e60b6f82d1b3e0f748492e3e5532 not found: ID does not exist"
Jul 29 20:53:39.579603 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579559    2581 scope.go:117] "RemoveContainer" containerID="6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"
Jul 29 20:53:39.579794 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579778    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e"} err="failed to get container status \"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e\": rpc error: code = NotFound desc = could not find container \"6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e\": container with ID starting with 6ac1513a9fd4b037d9791a3468d1f2a6ccf31bf4cf4fd2b441e2ec75d7ce3e6e not found: ID does not exist"
Jul 29 20:53:39.579883 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.579794    2581 scope.go:117] "RemoveContainer" containerID="2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"
Jul 29 20:53:39.580064 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.580044    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef"} err="failed to get container status \"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef\": rpc error: code = NotFound desc = could not find container \"2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef\": container with ID starting with 2f54625bb9b3e5ac2f8adb74bbdc60778ecc6c77ddb3c180c74147a23ae930ef not found: ID does not exist"
Jul 29 20:53:39.580115 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.580065    2581 scope.go:117] "RemoveContainer" containerID="cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"
Jul 29 20:53:39.580295 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.580280    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5"} err="failed to get container status \"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5\": rpc error: code = NotFound desc = could not find container \"cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5\": container with ID starting with cf2f696ce69ae89f0913e6eada7169484f3cebac68a8d86233039e0c1ec247e5 not found: ID does not exist"
Jul 29 20:53:39.580332 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.580296    2581 scope.go:117] "RemoveContainer" containerID="39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"
Jul 29 20:53:39.580543 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.580519    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955"} err="failed to get container status \"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955\": rpc error: code = NotFound desc = could not find container \"39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955\": container with ID starting with 39858a76bc068195cc57f88ba29be7f33cb2c2cc8862156ca2dba720f83e2955 not found: ID does not exist"
Jul 29 20:53:39.583337 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:39.583705 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583688    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="config-reloader"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583707    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="config-reloader"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583729    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="prom-label-proxy"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583738    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="prom-label-proxy"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583750    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-metric"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583759    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-metric"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583772    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583780    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy"
Jul 29 20:53:39.583788 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583790    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-web"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583798    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-web"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="alertmanager"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583821    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="alertmanager"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583833    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="init-config-reloader"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583841    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="init-config-reloader"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583918    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-web"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583931    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583942    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="kube-rbac-proxy-metric"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583952    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="alertmanager"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583961    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="prom-label-proxy"
Jul 29 20:53:39.584152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.583972    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" containerName="config-reloader"
Jul 29 20:53:39.589631 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.589614    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.592130 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.591927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 20:53:39.592130 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.591970    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 20:53:39.592130 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592007    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 20:53:39.592130 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 20:53:39.592428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592264    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 20:53:39.592428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 20:53:39.592428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592373    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vnjsd\""
Jul 29 20:53:39.592428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592423    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 20:53:39.592645 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.592329    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 20:53:39.598958 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.598938    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 20:53:39.602720 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.602697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:39.750194 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750194 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-config-out\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-web-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750681 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750681 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750681 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750681 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-config-volume\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.750681 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.750605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cpj5\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-kube-api-access-9cpj5\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852004 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.851966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cpj5\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-kube-api-access-9cpj5\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852195 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852195 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852322 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852504 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-config-out\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-web-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852634 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.852984 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.852958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.853359 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.853476 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.853476 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.854643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.854643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.854643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.853964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-config-volume\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.854943 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.854915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6da291d9-4d52-4900-b61b-88ba7b397463-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.856343 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.856314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6da291d9-4d52-4900-b61b-88ba7b397463-config-out\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.857016 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.856834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.858838 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.857109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-web-config\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.858838 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.858501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.861303 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.861243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-config-volume\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.862394 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.861519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.863127 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.863108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.863279 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.863250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.863362 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.863346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cpj5\" (UniqueName: \"kubernetes.io/projected/6da291d9-4d52-4900-b61b-88ba7b397463-kube-api-access-9cpj5\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.864317 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.864295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6da291d9-4d52-4900-b61b-88ba7b397463-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6da291d9-4d52-4900-b61b-88ba7b397463\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:39.902117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:39.902088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 20:53:40.046117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:40.046086    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 20:53:40.049698 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:53:40.049670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da291d9_4d52_4900_b61b_88ba7b397463.slice/crio-55e111f8472e6bb4da4e29b311d23c86c3f169c1f7e38b9b145ad6ebee149a6b WatchSource:0}: Error finding container 55e111f8472e6bb4da4e29b311d23c86c3f169c1f7e38b9b145ad6ebee149a6b: Status 404 returned error can't find the container with id 55e111f8472e6bb4da4e29b311d23c86c3f169c1f7e38b9b145ad6ebee149a6b
Jul 29 20:53:40.262500 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:40.262470    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9bbf1f2-54dc-416d-841b-0895d96adeba" path="/var/lib/kubelet/pods/a9bbf1f2-54dc-416d-841b-0895d96adeba/volumes"
Jul 29 20:53:40.528775 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:40.528740    2581 generic.go:356] "Generic (PLEG): container finished" podID="6da291d9-4d52-4900-b61b-88ba7b397463" containerID="dad51c76c490fd85f22b837611672e9bb12137c6a870b694105e21f7888dbcff" exitCode=0
Jul 29 20:53:40.529134 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:40.528816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerDied","Data":"dad51c76c490fd85f22b837611672e9bb12137c6a870b694105e21f7888dbcff"}
Jul 29 20:53:40.529134 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:40.528883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"55e111f8472e6bb4da4e29b311d23c86c3f169c1f7e38b9b145ad6ebee149a6b"}
Jul 29 20:53:41.536880 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.536848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"2985f48d8fe8748bd8cc3617b426ff6a5a083cf112c4c001f7950c7640041bba"}
Jul 29 20:53:41.537270 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.536909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"930ae1b83abc82173f059f3defd177064f9d9d1f989af11ff469e7f42e3f9228"}
Jul 29 20:53:41.537270 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.536937    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"63fefcbdef9b2f324cd252f4f78b6390f2ee18943a66de6b7facf3b43396cb87"}
Jul 29 20:53:41.537270 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.536959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"aea76c30ba70d2a5299a32d1bc78e31326faa80e931cb963a10b672dd52cf065"}
Jul 29 20:53:41.537270 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.536983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"6a2be26d397d02abdb3939a4a4fd8d17bfd69b803bf67356dd40e46e4a441451"}
Jul 29 20:53:41.537270 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.537006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6da291d9-4d52-4900-b61b-88ba7b397463","Type":"ContainerStarted","Data":"5f464eddf4849f1538d9ba7d212c3c94e0890a2b151b435f3a07953dc6eb3a9e"}
Jul 29 20:53:41.563532 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.563462    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.548589741 podStartE2EDuration="2.563440578s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.850837ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:53:41.561743619 +0000 UTC m=+371.810981134" watchObservedRunningTime="2026-07-29 20:53:41.563440578 +0000 UTC m=+371.812678094"
Jul 29 20:53:41.942775 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.942670    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:41.943076 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943042    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="prometheus" containerID="cri-o://2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363" gracePeriod=600
Jul 29 20:53:41.943490 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943457    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5" gracePeriod=600
Jul 29 20:53:41.943615 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943594    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy" containerID="cri-o://89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569" gracePeriod=600
Jul 29 20:53:41.943702 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943684    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-web" containerID="cri-o://2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251" gracePeriod=600
Jul 29 20:53:41.943782 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943763    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="thanos-sidecar" containerID="cri-o://80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00" gracePeriod=600
Jul 29 20:53:41.943860 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:41.943842    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="config-reloader" containerID="cri-o://7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c" gracePeriod=600
Jul 29 20:53:42.208125 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.208048    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.379830 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379839    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtq7d\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379886    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379946    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.379976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380355 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.380355 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380349    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.380537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380539    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380570    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380617    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380643    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380705    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380996 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380737    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380996 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls\") pod \"241f12bf-fee9-4d97-8d8c-ac245499282c\" (UID: \"241f12bf-fee9-4d97-8d8c-ac245499282c\") "
Jul 29 20:53:42.380996 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.380908    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:53:42.382715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.381144    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.382715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.381175    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.382715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.381191    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-db\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.382715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.381930    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.382715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.382164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.383049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.382772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.383049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.382794    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.383049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.382891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d" (OuterVolumeSpecName: "kube-api-access-rtq7d") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "kube-api-access-rtq7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:53:42.383049 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.382976    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.383294 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.383266    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.383565 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.383540    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config" (OuterVolumeSpecName: "config") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.383565 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.383554    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.383803 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.383752    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:53:42.383803 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.383763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.384494 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.384469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.385455 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.385423    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:53:42.385657 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.385635    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out" (OuterVolumeSpecName: "config-out") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:53:42.385924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.385898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.388446 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.388421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.397300 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.397281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config" (OuterVolumeSpecName: "web-config") pod "241f12bf-fee9-4d97-8d8c-ac245499282c" (UID: "241f12bf-fee9-4d97-8d8c-ac245499282c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482081    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-kube-rbac-proxy\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482120    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-metrics-client-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482130    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482141    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482150    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-grpc-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482158    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtq7d\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-kube-api-access-rtq7d\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482167    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482174 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482176    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482187    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-config\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482196    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482203    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/241f12bf-fee9-4d97-8d8c-ac245499282c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482213    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482223    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-metrics-client-certs\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482230    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-web-config\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482238    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241f12bf-fee9-4d97-8d8c-ac245499282c-tls-assets\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482247    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241f12bf-fee9-4d97-8d8c-ac245499282c-config-out\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.482540 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.482254    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/241f12bf-fee9-4d97-8d8c-ac245499282c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:53:42.543477 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543443    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5" exitCode=0
Jul 29 20:53:42.543477 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543467    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569" exitCode=0
Jul 29 20:53:42.543477 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543473    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251" exitCode=0
Jul 29 20:53:42.543477 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543479    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00" exitCode=0
Jul 29 20:53:42.543477 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543484    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c" exitCode=0
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543489    2581 generic.go:356] "Generic (PLEG): container finished" podID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363" exitCode=0
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543521    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543720    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"}
Jul 29 20:53:42.544001 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.543845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"241f12bf-fee9-4d97-8d8c-ac245499282c","Type":"ContainerDied","Data":"45c3bebe67165c4b0bf30a4dc401b583e06897667e92c473b6e27e70aeb2e583"}
Jul 29 20:53:42.552011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.551987    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.559354 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.559335    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.566255 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.566238    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.567581 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.567562    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:42.573846 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.573817    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.575282 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.575254    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:42.583664 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.583644    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.590963 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.590946    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.598010 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.597995    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.598282 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.598258    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.598340 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598290    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.598340 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598308    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.598598 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.598573    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.598635 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598609    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.598667 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598632    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.598857 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.598841    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.598899 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598862    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.598899 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.598877    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.599077 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.599060    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.599132 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599086    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.599132 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599108    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.599355 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.599340    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.599410 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599358    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.599410 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599370    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.599597 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.599583    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.599640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599600    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.599640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599613    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.599823 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:53:42.599805    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.599888 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599831    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.599888 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.599851    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.600120 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600078    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.600120 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600097    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.600377 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600353    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.600478 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600398    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.600694 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600666    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.600694 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.600693    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.601159 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601138    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.601225 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601160    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.601451 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601432    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.601517 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601452    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.601672 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601655    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.601716 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601683    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.601907 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601889    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.601950 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.601916    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.602170 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602145    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.602170 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602167    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.602296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:42.602430 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602409    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.602478 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602431    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.602546 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602532    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-thanos"
Jul 29 20:53:42.602604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602551    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-thanos"
Jul 29 20:53:42.602604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602566    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="thanos-sidecar"
Jul 29 20:53:42.602604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602576    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="thanos-sidecar"
Jul 29 20:53:42.602604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602587    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-web"
Jul 29 20:53:42.602604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602596    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-web"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602616    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="init-config-reloader"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602624    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="init-config-reloader"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602633    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="prometheus"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602638    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="prometheus"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602635    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602643    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="config-reloader"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602649    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="config-reloader"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602659    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602665    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602648    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602717    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-web"
Jul 29 20:53:42.602759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602761    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="prometheus"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602769    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="thanos-sidecar"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602792    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="config-reloader"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602799    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602805    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" containerName="kube-rbac-proxy-thanos"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.602997    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.603281 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.603021    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.603640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.603326    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.603640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.603359    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.603747 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.603663    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.603747 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.603690    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.604291 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604271    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.604291 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604291    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.604546 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604524    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.604608 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604546    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.604769 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604748    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.604806 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604770    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.604995 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604974    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.604995 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.604993    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.605207 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605191    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.605253 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605207    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.605427 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605408    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.605483 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605426    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.605659 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605642    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.605710 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605659    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.605882 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605864    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.605927 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.605884    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.606109 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606090    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.606155 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606109    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.606327 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606310    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.606370 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606330    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.606541 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606520    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.606583 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606543    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.606727 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606710    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.606763 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606727    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.606946 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606928    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.607005 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.606971    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.607214 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.607189    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.607298 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.607214    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.607534 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.607511    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.607598 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.607537    2581 scope.go:117] "RemoveContainer" containerID="baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"
Jul 29 20:53:42.607811 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.607766    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5"} err="failed to get container status \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": rpc error: code = NotFound desc = could not find container \"baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5\": container with ID starting with baac68fce7460ca19114bbd9545e767c063d1ac470a2eabc534fbea0c99c2fd5 not found: ID does not exist"
Jul 29 20:53:42.608473 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.608343    2581 scope.go:117] "RemoveContainer" containerID="89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"
Jul 29 20:53:42.608745 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.608722    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569"} err="failed to get container status \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": rpc error: code = NotFound desc = could not find container \"89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569\": container with ID starting with 89e6cc597dac98d6dbe35851aecb8dcff55fd631d62dd2ed8b8c6622793a5569 not found: ID does not exist"
Jul 29 20:53:42.608745 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.608746    2581 scope.go:117] "RemoveContainer" containerID="2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"
Jul 29 20:53:42.609084 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.609069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.609648 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.609561    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251"} err="failed to get container status \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": rpc error: code = NotFound desc = could not find container \"2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251\": container with ID starting with 2d08d73e6dd2a480a355786e4f23e8f06ebc249ba6cd8add4f45070cacf09251 not found: ID does not exist"
Jul 29 20:53:42.609648 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.609586    2581 scope.go:117] "RemoveContainer" containerID="80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"
Jul 29 20:53:42.610318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610193    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00"} err="failed to get container status \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": rpc error: code = NotFound desc = could not find container \"80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00\": container with ID starting with 80637f0bdf2346bf6cefd81a2ef82bd508cb7502e61719e9efd8e1deabce3a00 not found: ID does not exist"
Jul 29 20:53:42.610459 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610336    2581 scope.go:117] "RemoveContainer" containerID="7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"
Jul 29 20:53:42.610679 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610657    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c"} err="failed to get container status \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": rpc error: code = NotFound desc = could not find container \"7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c\": container with ID starting with 7b6c9993abf4946cc96958ce207aa3cf193a9b6ce4eca9946a203ae997fb611c not found: ID does not exist"
Jul 29 20:53:42.610771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610681    2581 scope.go:117] "RemoveContainer" containerID="2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"
Jul 29 20:53:42.610987 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610967    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363"} err="failed to get container status \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": rpc error: code = NotFound desc = could not find container \"2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363\": container with ID starting with 2915cccc9830320090131cae2a9a98a13435f96bc19f9f99d11cd7b99928e363 not found: ID does not exist"
Jul 29 20:53:42.611031 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.610989    2581 scope.go:117] "RemoveContainer" containerID="afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"
Jul 29 20:53:42.611224 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.611207    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7"} err="failed to get container status \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": rpc error: code = NotFound desc = could not find container \"afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7\": container with ID starting with afe2ed4bc667007bec9e7ddf3596e020d566b461c898353239152aa92504d7a7 not found: ID does not exist"
Jul 29 20:53:42.612011 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.611995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 20:53:42.612076 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612053    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 20:53:42.612186 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612170    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 20:53:42.612534 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612515    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 20:53:42.612607 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612562    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 20:53:42.612815 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612799    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 20:53:42.612880 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.612836    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 20:53:42.613100 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 20:53:42.613238 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613219    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5rafhlac0ss2v\""
Jul 29 20:53:42.613335 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613323    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 20:53:42.613433 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 20:53:42.613608 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613589    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 20:53:42.613704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613612    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 20:53:42.613704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 20:53:42.613704 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.613590    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cs2s5\""
Jul 29 20:53:42.615646 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.615627    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 20:53:42.618461 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.618443    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 20:53:42.623058 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.623038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:42.683262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683469 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-web-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683623 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683793 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683765    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-config-out\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvff\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-kube-api-access-hjvff\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.683955 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.683866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785072 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785324 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785324 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785324 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-web-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785324 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785324 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-config-out\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.785604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvff\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-kube-api-access-hjvff\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785822    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786045 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.785849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786814 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.786548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786814 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.786570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.786982 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.786901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.787161 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.787136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.787588 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.787563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.788283 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.788209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-web-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.788459 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.788346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.788459 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.788373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-config\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.788812 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.788706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.789753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.789020    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.789753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.789485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.789753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.789541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5cfd4640-d800-4271-9dc6-acfae561cd01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.789753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.789677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.790132 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.790108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.790509 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.790487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.790935 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.790916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.791167 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.791148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.791167 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.791157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5cfd4640-d800-4271-9dc6-acfae561cd01-config-out\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.791739 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.791723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5cfd4640-d800-4271-9dc6-acfae561cd01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.797268 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.797250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvff\" (UniqueName: \"kubernetes.io/projected/5cfd4640-d800-4271-9dc6-acfae561cd01-kube-api-access-hjvff\") pod \"prometheus-k8s-0\" (UID: \"5cfd4640-d800-4271-9dc6-acfae561cd01\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:42.920302 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:42.920268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:43.063843 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:43.063667    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:53:43.068589 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:53:43.068563    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfd4640_d800_4271_9dc6_acfae561cd01.slice/crio-7402e3b0587e813a6e32176a9cca986f4efe222ef71a81b119c9255af3c4f256 WatchSource:0}: Error finding container 7402e3b0587e813a6e32176a9cca986f4efe222ef71a81b119c9255af3c4f256: Status 404 returned error can't find the container with id 7402e3b0587e813a6e32176a9cca986f4efe222ef71a81b119c9255af3c4f256
Jul 29 20:53:43.550979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:43.550948    2581 generic.go:356] "Generic (PLEG): container finished" podID="5cfd4640-d800-4271-9dc6-acfae561cd01" containerID="415d9434455098a32a05f0a689d54794bb98decb9a0e65caf2d09ac19b832dbe" exitCode=0
Jul 29 20:53:43.551444 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:43.551000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerDied","Data":"415d9434455098a32a05f0a689d54794bb98decb9a0e65caf2d09ac19b832dbe"}
Jul 29 20:53:43.551444 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:43.551052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"7402e3b0587e813a6e32176a9cca986f4efe222ef71a81b119c9255af3c4f256"}
Jul 29 20:53:44.197715 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.197623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:53:44.200087 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.200066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f12a9-0f65-492a-8764-a3d0ace547fe-metrics-certs\") pod \"network-metrics-daemon-ldb4s\" (UID: \"8b7f12a9-0f65-492a-8764-a3d0ace547fe\") " pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:53:44.263121 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.263083    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241f12bf-fee9-4d97-8d8c-ac245499282c" path="/var/lib/kubelet/pods/241f12bf-fee9-4d97-8d8c-ac245499282c/volumes"
Jul 29 20:53:44.460513 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.460424    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vp87c\""
Jul 29 20:53:44.468391 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.468356    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldb4s"
Jul 29 20:53:44.558834 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.558798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"6e7be41594c775f53a86dee1accd9f249835edd7d432f2e75852bad21cfedb97"}
Jul 29 20:53:44.559212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.558888    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"d59217dc1341def5ba1af08834f897fde3dd866aae579a8934d4737dcacbc669"}
Jul 29 20:53:44.559212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.558931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"fd54f5f25491ac9707480dfa3edaad15b8d048240a8f18d5b918e2db9c0f0aa5"}
Jul 29 20:53:44.559212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.558977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"34642ec2c61c0f88663a162b5412430e5ece34109dc33617fc1353ebbc894405"}
Jul 29 20:53:44.559212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.559018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"6c31e786bab21a505d6f2bedc20f08fb4467b2f409cc5b712d12d135a1e9af9d"}
Jul 29 20:53:44.559212 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.559064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5cfd4640-d800-4271-9dc6-acfae561cd01","Type":"ContainerStarted","Data":"cf6f20b8e39e67d143b6b12ddd333b74caa6f45b3700cf12acd9efeb88e8de91"}
Jul 29 20:53:44.593140 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.593088    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.575514871 podStartE2EDuration="2.593070125s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.555254ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:53:44.589872445 +0000 UTC m=+374.839109973" watchObservedRunningTime="2026-07-29 20:53:44.593070125 +0000 UTC m=+374.842307640"
Jul 29 20:53:44.620227 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.620147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldb4s"]
Jul 29 20:53:44.622892 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:53:44.622866    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7f12a9_0f65_492a_8764_a3d0ace547fe.slice/crio-c1f657a21d4da5481c58b39c71cd28b6cbc3886488eb76671f243f69100bf391 WatchSource:0}: Error finding container c1f657a21d4da5481c58b39c71cd28b6cbc3886488eb76671f243f69100bf391: Status 404 returned error can't find the container with id c1f657a21d4da5481c58b39c71cd28b6cbc3886488eb76671f243f69100bf391
Jul 29 20:53:44.624764 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:44.624749    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:53:45.565949 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:45.565917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldb4s" event={"ID":"8b7f12a9-0f65-492a-8764-a3d0ace547fe","Type":"ContainerStarted","Data":"c1f657a21d4da5481c58b39c71cd28b6cbc3886488eb76671f243f69100bf391"}
Jul 29 20:53:45.832671 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:45.832578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:45.893824 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:45.893786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:46.571733 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.571695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldb4s" event={"ID":"8b7f12a9-0f65-492a-8764-a3d0ace547fe","Type":"ContainerStarted","Data":"649db47ec5ee41b7ad5932e0ed90221b91c469fbd89a39197fdbef740c6a5613"}
Jul 29 20:53:46.572095 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.571752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldb4s" event={"ID":"8b7f12a9-0f65-492a-8764-a3d0ace547fe","Type":"ContainerStarted","Data":"ff137e3536c1b7bb3baaaeb5d2c36e6461bf347abe277827c75721bcc0207d70"}
Jul 29 20:53:46.587019 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.586971    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ldb4s" podStartSLOduration=374.66659749 podStartE2EDuration="6m15.586955695s" totalImagesPullingTime="920.358205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:53:46.586719109 +0000 UTC m=+376.835956626" watchObservedRunningTime="2026-07-29 20:53:46.586955695 +0000 UTC m=+376.836193211"
Jul 29 20:53:46.606759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.606718    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:46.660855 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.660786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:46.727866 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.727813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:46.984229 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:46.984123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:47.091536 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:47.091490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:47.813447 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:47.813411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:47.870318 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:47.870278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:47.920925 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:47.920892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:53:47.985454 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:47.985424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:48.375428 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:48.375364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:48.425575 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:48.425531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:48.659363 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:48.659270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:48.759931 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:48.759893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:48.814581 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:53:48.814536    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:00.856010 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:00.855971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:00.958696 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:00.958659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:24.057585 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:24.057551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:24.113717 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:24.113690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:38.979026 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:38.978989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:39.077519 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:39.077475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:42.920867 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:42.920829    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:54:42.937093 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:42.937068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:54:43.812266 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:54:43.812239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:55:00.385878 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:00.385791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:01.764906 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:01.764849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:03.995875 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:03.995836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:04.100850 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:04.100817    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:06.928047 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:06.928010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:07.029924 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:07.029893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:07.190447 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:07.190343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:07.296283 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:07.296254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:08.940673 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:08.940638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:09.382781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:09.382747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:10.593368 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:10.593332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:10.693617 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:10.693587    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:14.127458 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:14.127419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:14.186656 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:14.186625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:17.748083 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:17.748041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:33.320976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:33.320945    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:33.372178 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:33.372143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:33.564724 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:33.564687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:33.663030 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:33.662940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:35.295499 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.295466    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"]
Jul 29 20:55:35.299498 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.299472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.301585 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.301561    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:55:35.302285 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.302263    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 20:55:35.302411 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.302313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4b758\""
Jul 29 20:55:35.308502 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.308479    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"]
Jul 29 20:55:35.373975 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.373940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/199f1e55-68cc-4d10-afe5-9c97456a94bf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.374166 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.373981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvlcp\" (UniqueName: \"kubernetes.io/projected/199f1e55-68cc-4d10-afe5-9c97456a94bf-kube-api-access-nvlcp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.475115 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.475080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/199f1e55-68cc-4d10-afe5-9c97456a94bf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.475322 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.475122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvlcp\" (UniqueName: \"kubernetes.io/projected/199f1e55-68cc-4d10-afe5-9c97456a94bf-kube-api-access-nvlcp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.475553 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.475529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/199f1e55-68cc-4d10-afe5-9c97456a94bf-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.489047 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.489016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvlcp\" (UniqueName: \"kubernetes.io/projected/199f1e55-68cc-4d10-afe5-9c97456a94bf-kube-api-access-nvlcp\") pod \"openshift-lws-operator-fd8ccff4c-l94c4\" (UID: \"199f1e55-68cc-4d10-afe5-9c97456a94bf\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.633939 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.633852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"
Jul 29 20:55:35.769190 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:35.769167    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4"]
Jul 29 20:55:35.774851 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:55:35.774823    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod199f1e55_68cc_4d10_afe5_9c97456a94bf.slice/crio-ee5df5408dbcdfce77e49b524355193d72ce7c2a675c4086503b9f8a44384644 WatchSource:0}: Error finding container ee5df5408dbcdfce77e49b524355193d72ce7c2a675c4086503b9f8a44384644: Status 404 returned error can't find the container with id ee5df5408dbcdfce77e49b524355193d72ce7c2a675c4086503b9f8a44384644
Jul 29 20:55:36.012460 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:36.012426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4" event={"ID":"199f1e55-68cc-4d10-afe5-9c97456a94bf","Type":"ContainerStarted","Data":"ee5df5408dbcdfce77e49b524355193d72ce7c2a675c4086503b9f8a44384644"}
Jul 29 20:55:39.029457 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:39.029424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4" event={"ID":"199f1e55-68cc-4d10-afe5-9c97456a94bf","Type":"ContainerStarted","Data":"b66bd58b980851fac23a8f7c82ea6b9da6eac21d2b7fb32718cd42c717d66e89"}
Jul 29 20:55:39.046057 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:39.046010    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l94c4" podStartSLOduration=1.3769548870000001 podStartE2EDuration="4.045993564s" totalImagesPullingTime="2.669038677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:55:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:55:39.0438455 +0000 UTC m=+489.293083017" watchObservedRunningTime="2026-07-29 20:55:39.045993564 +0000 UTC m=+489.295231079"
Jul 29 20:55:39.758817 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:39.758774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:39.858976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:39.858942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:41.564175 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:41.564134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:49.825229 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.825143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:49.888827 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.888794    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"]
Jul 29 20:55:49.892504 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.892487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:49.894753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.894732    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 20:55:49.894753 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.894745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 20:55:49.894907 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.894783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5qk7z\""
Jul 29 20:55:49.894907 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.894814    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 20:55:49.900612 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.900592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"]
Jul 29 20:55:49.904671 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:49.904646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:50.005013 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.004979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-metrics-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.005190 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.005049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ab98a1-3774-4d92-89f2-b1f45b0369b9-manager-config\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.005190 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.005090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv6hz\" (UniqueName: \"kubernetes.io/projected/37ab98a1-3774-4d92-89f2-b1f45b0369b9-kube-api-access-rv6hz\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.005190 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.005150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.106005 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.105917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ab98a1-3774-4d92-89f2-b1f45b0369b9-manager-config\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.106005 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.105959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv6hz\" (UniqueName: \"kubernetes.io/projected/37ab98a1-3774-4d92-89f2-b1f45b0369b9-kube-api-access-rv6hz\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.106211 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.106028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.106211 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.106060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-metrics-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.106702 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.106682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ab98a1-3774-4d92-89f2-b1f45b0369b9-manager-config\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.108636 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.108615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-metrics-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.108740 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.108713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ab98a1-3774-4d92-89f2-b1f45b0369b9-cert\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.120326 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.120305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv6hz\" (UniqueName: \"kubernetes.io/projected/37ab98a1-3774-4d92-89f2-b1f45b0369b9-kube-api-access-rv6hz\") pod \"lws-controller-manager-64fb5f9c89-gwtft\" (UID: \"37ab98a1-3774-4d92-89f2-b1f45b0369b9\") " pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.202353 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.202312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:50.367316 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:50.367083    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"]
Jul 29 20:55:50.369903 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:55:50.369876    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ab98a1_3774_4d92_89f2_b1f45b0369b9.slice/crio-40ce59ce7b98286cae7cf9780e7fc78d17d35450b4a6bb1d398ec88c48d5d531 WatchSource:0}: Error finding container 40ce59ce7b98286cae7cf9780e7fc78d17d35450b4a6bb1d398ec88c48d5d531: Status 404 returned error can't find the container with id 40ce59ce7b98286cae7cf9780e7fc78d17d35450b4a6bb1d398ec88c48d5d531
Jul 29 20:55:51.081907 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:51.081867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft" event={"ID":"37ab98a1-3774-4d92-89f2-b1f45b0369b9","Type":"ContainerStarted","Data":"40ce59ce7b98286cae7cf9780e7fc78d17d35450b4a6bb1d398ec88c48d5d531"}
Jul 29 20:55:52.088941 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:52.088856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft" event={"ID":"37ab98a1-3774-4d92-89f2-b1f45b0369b9","Type":"ContainerStarted","Data":"1dcc9f7430009f76132e0863e975dacd4f8ba15e196dce27ba2483699e0963e0"}
Jul 29 20:55:52.089315 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:52.088978    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:55:52.107975 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:52.107928    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft" podStartSLOduration=1.639297015 podStartE2EDuration="3.107913907s" totalImagesPullingTime="1.468616892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:55:52.105931231 +0000 UTC m=+502.355168761" watchObservedRunningTime="2026-07-29 20:55:52.107913907 +0000 UTC m=+502.357151422"
Jul 29 20:55:54.517888 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:54.517852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:54.623022 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:54.622986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.121731 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.121687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.360756 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.360727    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.460964 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.460861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.512258 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.512228    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"]
Jul 29 20:55:55.515619 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.515594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.518070 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.518048    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 20:55:55.518070 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.518060    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f44g6\""
Jul 29 20:55:55.518587 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.518297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 20:55:55.518587 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.518345    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 20:55:55.518587 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.518443    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 20:55:55.531773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.531752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"]
Jul 29 20:55:55.615222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.615186    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.660364 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.660332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-tmp\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.660364 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.660366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.660581 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.660470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b4fw\" (UniqueName: \"kubernetes.io/projected/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-kube-api-access-8b4fw\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.660581 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.660499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-manifests\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.660648 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.660583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.715979 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.715886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.761984 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.761954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b4fw\" (UniqueName: \"kubernetes.io/projected/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-kube-api-access-8b4fw\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.762117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.761989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-manifests\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.762117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.762044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.762117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.762067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-tmp\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.762117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.762087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.762466 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.762446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-manifests\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.764538 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.764509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-tmp\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.764785 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.764765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-webhook-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.764888 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.764870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.770680 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.770655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b4fw\" (UniqueName: \"kubernetes.io/projected/ad9d1392-ed53-4e83-a4b4-155ff85abaf1-kube-api-access-8b4fw\") pod \"opendatahub-operator-controller-manager-57645d8769-pwxhj\" (UID: \"ad9d1392-ed53-4e83-a4b4-155ff85abaf1\") " pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.819137 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.819097    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.827028 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.827009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:55:55.926305 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.926161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:55.968259 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:55.968190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"]
Jul 29 20:55:55.971746 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:55:55.971719    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9d1392_ed53_4e83_a4b4_155ff85abaf1.slice/crio-0ffa57896f2dc76d6635b9a0b5eb5de02790d47ad6587b06e4cc6b5680897acf WatchSource:0}: Error finding container 0ffa57896f2dc76d6635b9a0b5eb5de02790d47ad6587b06e4cc6b5680897acf: Status 404 returned error can't find the container with id 0ffa57896f2dc76d6635b9a0b5eb5de02790d47ad6587b06e4cc6b5680897acf
Jul 29 20:55:56.018612 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:56.018576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:56.107214 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:56.107181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj" event={"ID":"ad9d1392-ed53-4e83-a4b4-155ff85abaf1","Type":"ContainerStarted","Data":"0ffa57896f2dc76d6635b9a0b5eb5de02790d47ad6587b06e4cc6b5680897acf"}
Jul 29 20:55:56.129421 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:56.129372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:59.123976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:55:59.123941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj" event={"ID":"ad9d1392-ed53-4e83-a4b4-155ff85abaf1","Type":"ContainerStarted","Data":"2bd86326ed7973301962aa346187c464c910d65c9b0b3e01aa443e5ff02f7924"}
Jul 29 20:56:01.134308 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:01.134272    2581 generic.go:356] "Generic (PLEG): container finished" podID="ad9d1392-ed53-4e83-a4b4-155ff85abaf1" containerID="2bd86326ed7973301962aa346187c464c910d65c9b0b3e01aa443e5ff02f7924" exitCode=0
Jul 29 20:56:01.134702 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:01.134313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj" event={"ID":"ad9d1392-ed53-4e83-a4b4-155ff85abaf1","Type":"ContainerDied","Data":"2bd86326ed7973301962aa346187c464c910d65c9b0b3e01aa443e5ff02f7924"}
Jul 29 20:56:02.140982 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:02.140948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj" event={"ID":"ad9d1392-ed53-4e83-a4b4-155ff85abaf1","Type":"ContainerStarted","Data":"5e6bb03c9909fcf4bd7976ed72fcc52a90873c096f7315214197c26f352ab4c5"}
Jul 29 20:56:02.141335 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:02.141183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:56:02.164500 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:02.164451    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj" podStartSLOduration=2.588924392 podStartE2EDuration="7.164436345s" totalImagesPullingTime="3.498704864s" totalInitContainerRuntime="1.076807089s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:55:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:56:02.162239685 +0000 UTC m=+512.411477210" watchObservedRunningTime="2026-07-29 20:56:02.164436345 +0000 UTC m=+512.413673861"
Jul 29 20:56:03.096321 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:03.096288    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64fb5f9c89-gwtft"
Jul 29 20:56:07.558295 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:07.558230    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:07.657671 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:07.657632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:13.148340 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:13.148312    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57645d8769-pwxhj"
Jul 29 20:56:14.839122 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:14.839081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:14.969580 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:14.969542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:15.373737 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:15.373696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:15.604533 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:15.604501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:15.697247 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:15.697157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:15.792077 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:15.792043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:16.132591 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:16.132555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:16.449991 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:16.449903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:16.878858 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:16.878810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:16.942224 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:16.942189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:18.008539 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:18.008506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:21.216706 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:21.216575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:22.707971 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.707937    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"]
Jul 29 20:56:22.710367 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.710346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.712833 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.712626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 20:56:22.712833 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.712668    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 20:56:22.712833 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.712697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rtc4z\""
Jul 29 20:56:22.712833 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.712626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 20:56:22.712833 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.712626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 20:56:22.723296 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.723273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"]
Jul 29 20:56:22.809565 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e973c5ca-5b28-4619-8909-4055f1feb70c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809565 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tlzb\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-kube-api-access-8tlzb\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.809818 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.809755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910545 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e973c5ca-5b28-4619-8909-4055f1feb70c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910545 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tlzb\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-kube-api-access-8tlzb\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.910781 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.910714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.911263 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.911235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.913221 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.913201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.913418 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.913283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.913507 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.913464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e973c5ca-5b28-4619-8909-4055f1feb70c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.913507 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.913498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.930708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.930685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:22.930942 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:22.930922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tlzb\" (UniqueName: \"kubernetes.io/projected/e973c5ca-5b28-4619-8909-4055f1feb70c-kube-api-access-8tlzb\") pod \"istiod-openshift-gateway-5677ccb49b-pnhjv\" (UID: \"e973c5ca-5b28-4619-8909-4055f1feb70c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:23.020292 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:23.020263    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:23.180328 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:23.180290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:23.190468 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:23.190444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"]
Jul 29 20:56:23.192947 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:56:23.192918    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode973c5ca_5b28_4619_8909_4055f1feb70c.slice/crio-c88e5f9b7878c85b90bc79045247e7cf804a45df8cf844d16b0f1e4eed4ff34e WatchSource:0}: Error finding container c88e5f9b7878c85b90bc79045247e7cf804a45df8cf844d16b0f1e4eed4ff34e: Status 404 returned error can't find the container with id c88e5f9b7878c85b90bc79045247e7cf804a45df8cf844d16b0f1e4eed4ff34e
Jul 29 20:56:23.237095 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:23.237063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv" event={"ID":"e973c5ca-5b28-4619-8909-4055f1feb70c","Type":"ContainerStarted","Data":"c88e5f9b7878c85b90bc79045247e7cf804a45df8cf844d16b0f1e4eed4ff34e"}
Jul 29 20:56:23.276713 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:23.276617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:24.131258 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:24.131211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:24.380963 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:24.380903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:24.529897 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:24.529809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:25.736915 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:25.736816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:25.837476 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:25.837429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:26.011246 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.011205    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 20:56:26.011393 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.011285    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 29 20:56:26.066032 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.065991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:26.264537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.264454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:26.264537 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.264497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv" event={"ID":"e973c5ca-5b28-4619-8909-4055f1feb70c","Type":"ContainerStarted","Data":"bb9dfcce77f42e17b941a81e84e33123910643d38390540970fa59da65a30962"}
Jul 29 20:56:26.310438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.310324    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv" podStartSLOduration=1.4943686409999999 podStartE2EDuration="4.310304543s" totalImagesPullingTime="2.815935902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:56:26.309559966 +0000 UTC m=+536.558797481" watchObservedRunningTime="2026-07-29 20:56:26.310304543 +0000 UTC m=+536.559542057"
Jul 29 20:56:26.446291 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:26.446255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:27.068898 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:27.068862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:27.170606 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:27.170568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:27.265695 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:27.265667    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pnhjv"
Jul 29 20:56:27.271640 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:27.271611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:41.882544 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:56:41.882505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:09.474889 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:09.474855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:16.264346 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:16.264300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:16.359994 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:16.359957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:16.475643 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:16.475608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:30.568254 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:30.568221    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:57:30.570914 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:30.568525    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 20:57:35.383304 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:35.383270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:35.943071 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:35.943039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:37.481873 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:37.481837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:45.385732 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:45.385684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:45.918618 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:45.918583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:46.017764 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:46.017732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:46.118680 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:46.118648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:46.218734 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:46.218637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:48.992854 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:48.992813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:52.036789 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:52.036698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:53.480251 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:53.480216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:53.791780 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:53.791740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:54.390617 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:54.390582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:56.827895 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:56.827858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:56.919478 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:56.919447    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:57.794554 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:57.794524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:59.650497 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:57:59.650457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:01.201664 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:01.201627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:01.305827 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:01.305776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:03.626542 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:03.626508    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:03.959773 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:03.959686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:04.777393 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:04.777357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:04.843464 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:04.843437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:05.087922 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:05.087840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:14.025821 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:14.025781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:14.122545 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:14.122505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:17.068532 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:17.068495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:17.161143 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:17.161109    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:19.590611 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:19.590577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:19.691923 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:19.691893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:23.592326 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:23.592291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:23.691326 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:23.691278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:32.607364 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:32.607322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:32.705877 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:32.705848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:32.805438 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:32.805374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:32.865122 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:32.865027    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:35.407917 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:35.407868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:35.510271 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:35.510239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:42.320422 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:42.320368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:42.607784 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:42.607708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:44.199093 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:44.199053    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:44.299335 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:44.299294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:44.879257 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:44.879227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:44.940976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:44.940942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:45.794712 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.794677    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:58:45.798333 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.798316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:45.799910 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.799878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:45.800403 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.800361    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 20:58:45.800502 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.800485    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 20:58:45.800555 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.800499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hlkjq\""
Jul 29 20:58:45.800555 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.800501    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 20:58:45.807361 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.807341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:58:45.894866 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.894823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:58:45.922726 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.922693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:45.922902 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:45.922738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdtvz\" (UniqueName: \"kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.023720 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.023683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.023720 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.023725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdtvz\" (UniqueName: \"kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.024355 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.024329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.031994 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.031959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdtvz\" (UniqueName: \"kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz\") pod \"limitador-limitador-64c94fd64f-cvg7m\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.110332 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.110245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:46.241898 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.241863    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:58:46.244227 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:58:46.244202    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd793d7_d466_47d4_aa9c_239921e626f2.slice/crio-a9da5a76fa537ad381ee7b6cc38f425f57324b890b514712f7debcb2260c72a9 WatchSource:0}: Error finding container a9da5a76fa537ad381ee7b6cc38f425f57324b890b514712f7debcb2260c72a9: Status 404 returned error can't find the container with id a9da5a76fa537ad381ee7b6cc38f425f57324b890b514712f7debcb2260c72a9
Jul 29 20:58:46.245951 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.245888    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:58:46.897947 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:46.897918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" event={"ID":"4fd793d7-d466-47d4-aa9c-239921e626f2","Type":"ContainerStarted","Data":"a9da5a76fa537ad381ee7b6cc38f425f57324b890b514712f7debcb2260c72a9"}
Jul 29 20:58:47.073771 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:47.073686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:49.914871 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:49.914774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" event={"ID":"4fd793d7-d466-47d4-aa9c-239921e626f2","Type":"ContainerStarted","Data":"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"}
Jul 29 20:58:49.915243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:49.914897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:58:49.934894 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:49.934843    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" podStartSLOduration=2.134121809 podStartE2EDuration="4.934828974s" totalImagesPullingTime="2.800707165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:58:49.933276483 +0000 UTC m=+680.182513999" watchObservedRunningTime="2026-07-29 20:58:49.934828974 +0000 UTC m=+680.184066549"
Jul 29 20:58:56.395976 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:56.395936    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:56.452666 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:56.452628    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:58.207167 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:58:58.207130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:00.920554 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:00.920521    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:59:02.182736 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.182698    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:59:02.183248 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.182923    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" podUID="4fd793d7-d466-47d4-aa9c-239921e626f2" containerName="limitador" containerID="cri-o://30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5" gracePeriod=30
Jul 29 20:59:02.734065 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.734044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:59:02.778319 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.778290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file\") pod \"4fd793d7-d466-47d4-aa9c-239921e626f2\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") "
Jul 29 20:59:02.778319 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.778324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdtvz\" (UniqueName: \"kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz\") pod \"4fd793d7-d466-47d4-aa9c-239921e626f2\" (UID: \"4fd793d7-d466-47d4-aa9c-239921e626f2\") "
Jul 29 20:59:02.778707 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.778681    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file" (OuterVolumeSpecName: "config-file") pod "4fd793d7-d466-47d4-aa9c-239921e626f2" (UID: "4fd793d7-d466-47d4-aa9c-239921e626f2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:59:02.780654 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.780634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz" (OuterVolumeSpecName: "kube-api-access-qdtvz") pod "4fd793d7-d466-47d4-aa9c-239921e626f2" (UID: "4fd793d7-d466-47d4-aa9c-239921e626f2"). InnerVolumeSpecName "kube-api-access-qdtvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:59:02.879987 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.879948    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fd793d7-d466-47d4-aa9c-239921e626f2-config-file\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:59:02.879987 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.879979    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdtvz\" (UniqueName: \"kubernetes.io/projected/4fd793d7-d466-47d4-aa9c-239921e626f2-kube-api-access-qdtvz\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:59:02.978396 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.978349    2581 generic.go:356] "Generic (PLEG): container finished" podID="4fd793d7-d466-47d4-aa9c-239921e626f2" containerID="30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5" exitCode=0
Jul 29 20:59:02.978588 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.978411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"
Jul 29 20:59:02.978588 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.978440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" event={"ID":"4fd793d7-d466-47d4-aa9c-239921e626f2","Type":"ContainerDied","Data":"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"}
Jul 29 20:59:02.978588 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.978507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cvg7m" event={"ID":"4fd793d7-d466-47d4-aa9c-239921e626f2","Type":"ContainerDied","Data":"a9da5a76fa537ad381ee7b6cc38f425f57324b890b514712f7debcb2260c72a9"}
Jul 29 20:59:02.978588 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.978538    2581 scope.go:117] "RemoveContainer" containerID="30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"
Jul 29 20:59:02.987866 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.987848    2581 scope.go:117] "RemoveContainer" containerID="30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"
Jul 29 20:59:02.988153 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:02.988136    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5\": container with ID starting with 30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5 not found: ID does not exist" containerID="30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"
Jul 29 20:59:02.988197 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.988165    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5"} err="failed to get container status \"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5\": rpc error: code = NotFound desc = could not find container \"30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5\": container with ID starting with 30ac8bb925ec4d464eaa8dae0b29c0984ac76547c0e5dd5af19de1b8a35908e5 not found: ID does not exist"
Jul 29 20:59:02.998709 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:02.998686    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:59:03.004244 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:03.004223    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cvg7m"]
Jul 29 20:59:04.263266 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:04.263232    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd793d7-d466-47d4-aa9c-239921e626f2" path="/var/lib/kubelet/pods/4fd793d7-d466-47d4-aa9c-239921e626f2/volumes"
Jul 29 20:59:05.223371 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.223319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:05.324120 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.324086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:05.648733 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.648699    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"]
Jul 29 20:59:05.649152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.649132    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd793d7-d466-47d4-aa9c-239921e626f2" containerName="limitador"
Jul 29 20:59:05.649262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.649155    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd793d7-d466-47d4-aa9c-239921e626f2" containerName="limitador"
Jul 29 20:59:05.649262 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.649253    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd793d7-d466-47d4-aa9c-239921e626f2" containerName="limitador"
Jul 29 20:59:05.653222 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.653201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.655775 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.655757    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 20:59:05.656034 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.656018    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-r8hht\""
Jul 29 20:59:05.656581 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.656563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 20:59:05.656659 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.656567    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 20:59:05.659904 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.659877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"]
Jul 29 20:59:05.675431 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.675363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:05.705156 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.705123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552qp\" (UniqueName: \"kubernetes.io/projected/c8457d4f-7070-4d34-b1ef-225e5906418a-kube-api-access-552qp\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.705285 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.705181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8457d4f-7070-4d34-b1ef-225e5906418a-data\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.806541 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.806504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552qp\" (UniqueName: \"kubernetes.io/projected/c8457d4f-7070-4d34-b1ef-225e5906418a-kube-api-access-552qp\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.806708 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.806576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8457d4f-7070-4d34-b1ef-225e5906418a-data\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.806970 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.806953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c8457d4f-7070-4d34-b1ef-225e5906418a-data\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.815345 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.815308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-552qp\" (UniqueName: \"kubernetes.io/projected/c8457d4f-7070-4d34-b1ef-225e5906418a-kube-api-access-552qp\") pod \"postgres-69fb4594cf-wcn56\" (UID: \"c8457d4f-7070-4d34-b1ef-225e5906418a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:05.964778 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:05.964691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:06.099805 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:06.099775    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"]
Jul 29 20:59:06.103019 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:59:06.102990    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8457d4f_7070_4d34_b1ef_225e5906418a.slice/crio-2f4b83e151ce97df678157bc4359dd032d7d98d8ae933694bcefbd44863b6715 WatchSource:0}: Error finding container 2f4b83e151ce97df678157bc4359dd032d7d98d8ae933694bcefbd44863b6715: Status 404 returned error can't find the container with id 2f4b83e151ce97df678157bc4359dd032d7d98d8ae933694bcefbd44863b6715
Jul 29 20:59:07.003552 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:07.003443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56" event={"ID":"c8457d4f-7070-4d34-b1ef-225e5906418a","Type":"ContainerStarted","Data":"2f4b83e151ce97df678157bc4359dd032d7d98d8ae933694bcefbd44863b6715"}
Jul 29 20:59:11.026862 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:11.026825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56" event={"ID":"c8457d4f-7070-4d34-b1ef-225e5906418a","Type":"ContainerStarted","Data":"5c2e5e262ccf96b322ffdbeec1a15fa721eb8d2c805c4175596c7107d8b36f3c"}
Jul 29 20:59:11.027230 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:11.026887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:11.043243 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:11.043193    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56" podStartSLOduration=1.221908427 podStartE2EDuration="6.043175994s" totalImagesPullingTime="4.821267567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:59:11.041702231 +0000 UTC m=+701.290939758" watchObservedRunningTime="2026-07-29 20:59:11.043175994 +0000 UTC m=+701.292413512"
Jul 29 20:59:17.061178 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:17.061146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wcn56"
Jul 29 20:59:17.378138 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:17.378050    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:17.661152 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:17.661058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:23.812060 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.811979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:23.834709 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.834673    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:23.844150 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.844123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:23.846432 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.846374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 20:59:23.846567 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.846456    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 20:59:23.846567 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.846530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bt8mv\""
Jul 29 20:59:23.850320 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.850296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:23.891372 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.891325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:23.947531 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.947494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:23.978606 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.978573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:23.978761 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.978711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxxwj\" (UniqueName: \"kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:23.990377 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:23.990341    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 20:59:24.003493 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.003469    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 20:59:24.003655 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.003578    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.080056 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.079973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxxwj\" (UniqueName: \"kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.080056 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.080031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.080248 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:24.080144    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 20:59:24.080286 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:24.080269    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert podName:ec982344-6200-404b-8beb-9511642c0bff nodeName:}" failed. No retries permitted until 2026-07-29 20:59:24.580192026 +0000 UTC m=+714.829429520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert") pod "maas-controller-67cfd5c6dc-gsgdl" (UID: "ec982344-6200-404b-8beb-9511642c0bff") : secret "maas-controller-webhook-cert" not found
Jul 29 20:59:24.088944 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.088914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxxwj\" (UniqueName: \"kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.181203 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.181164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhj8\" (UniqueName: \"kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.181377 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.181253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.282689 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.282656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.282866 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.282765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhj8\" (UniqueName: \"kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.282866 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:24.282811    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 20:59:24.282939 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:24.282880    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert podName:f2ae6ec9-e765-4ec1-b102-d3a761793388 nodeName:}" failed. No retries permitted until 2026-07-29 20:59:24.782859669 +0000 UTC m=+715.032097177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert") pod "maas-controller-576968bbcc-6jfhm" (UID: "f2ae6ec9-e765-4ec1-b102-d3a761793388") : secret "maas-controller-webhook-cert" not found
Jul 29 20:59:24.291795 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.291763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhj8\" (UniqueName: \"kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.422160 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.422072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:24.586604 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.586573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.589117 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.589098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") pod \"maas-controller-67cfd5c6dc-gsgdl\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") " pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.756782 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.756745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:24.788038 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.788012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.790805 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.790781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") pod \"maas-controller-576968bbcc-6jfhm\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") " pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:24.903234 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.903194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:24.907680 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:59:24.907655    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec982344_6200_404b_8beb_9511642c0bff.slice/crio-19f340a39c62301ff05f0322c288b76516513be476c170dd3a9cdfdb8dfdccd3 WatchSource:0}: Error finding container 19f340a39c62301ff05f0322c288b76516513be476c170dd3a9cdfdb8dfdccd3: Status 404 returned error can't find the container with id 19f340a39c62301ff05f0322c288b76516513be476c170dd3a9cdfdb8dfdccd3
Jul 29 20:59:24.914899 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:24.914877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:25.071357 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:25.071320    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 20:59:25.074490 ip-10-0-135-229 kubenswrapper[2581]: W0729 20:59:25.074456    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ae6ec9_e765_4ec1_b102_d3a761793388.slice/crio-634e938432e99f5ac4ce1a533f0c580c14e3f845fbcabf9bccbfd31df909fc3f WatchSource:0}: Error finding container 634e938432e99f5ac4ce1a533f0c580c14e3f845fbcabf9bccbfd31df909fc3f: Status 404 returned error can't find the container with id 634e938432e99f5ac4ce1a533f0c580c14e3f845fbcabf9bccbfd31df909fc3f
Jul 29 20:59:25.092857 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:25.092821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-6jfhm" event={"ID":"f2ae6ec9-e765-4ec1-b102-d3a761793388","Type":"ContainerStarted","Data":"634e938432e99f5ac4ce1a533f0c580c14e3f845fbcabf9bccbfd31df909fc3f"}
Jul 29 20:59:25.094044 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:25.094020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" event={"ID":"ec982344-6200-404b-8beb-9511642c0bff","Type":"ContainerStarted","Data":"19f340a39c62301ff05f0322c288b76516513be476c170dd3a9cdfdb8dfdccd3"}
Jul 29 20:59:27.791009 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:27.790951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:27.889987 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:27.889898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 20:59:28.114572 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.114534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" event={"ID":"ec982344-6200-404b-8beb-9511642c0bff","Type":"ContainerStarted","Data":"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"}
Jul 29 20:59:28.114759 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.114636    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:28.115984 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.115960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-6jfhm" event={"ID":"f2ae6ec9-e765-4ec1-b102-d3a761793388","Type":"ContainerStarted","Data":"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"}
Jul 29 20:59:28.116109 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.116095    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:28.135630 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.135581    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" podStartSLOduration=2.429669674 podStartE2EDuration="5.135566122s" totalImagesPullingTime="2.705896448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:59:28.134337204 +0000 UTC m=+718.383574718" watchObservedRunningTime="2026-07-29 20:59:28.135566122 +0000 UTC m=+718.384803658"
Jul 29 20:59:28.158692 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:28.158592    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-576968bbcc-6jfhm" podStartSLOduration=2.626720781 podStartE2EDuration="5.158570918s" totalImagesPullingTime="2.531850137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:59:28.158276245 +0000 UTC m=+718.407513801" watchObservedRunningTime="2026-07-29 20:59:28.158570918 +0000 UTC m=+718.407808433"
Jul 29 20:59:39.124727 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.124697    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 20:59:39.125175 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.124762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:39.188064 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.188032    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:39.188289 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.188262    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" podUID="ec982344-6200-404b-8beb-9511642c0bff" containerName="manager" containerID="cri-o://2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d" gracePeriod=30
Jul 29 20:59:39.435972 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.435948    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:39.519000 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.518963    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxxwj\" (UniqueName: \"kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj\") pod \"ec982344-6200-404b-8beb-9511642c0bff\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") "
Jul 29 20:59:39.519195 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.519028    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") pod \"ec982344-6200-404b-8beb-9511642c0bff\" (UID: \"ec982344-6200-404b-8beb-9511642c0bff\") "
Jul 29 20:59:39.521290 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.521265    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ec982344-6200-404b-8beb-9511642c0bff" (UID: "ec982344-6200-404b-8beb-9511642c0bff"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:59:39.521401 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.521311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj" (OuterVolumeSpecName: "kube-api-access-rxxwj") pod "ec982344-6200-404b-8beb-9511642c0bff" (UID: "ec982344-6200-404b-8beb-9511642c0bff"). InnerVolumeSpecName "kube-api-access-rxxwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:59:39.619816 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.619774    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxxwj\" (UniqueName: \"kubernetes.io/projected/ec982344-6200-404b-8beb-9511642c0bff-kube-api-access-rxxwj\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:59:39.619816 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:39.619813    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec982344-6200-404b-8beb-9511642c0bff-webhook-cert\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 20:59:40.175036 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.175001    2581 generic.go:356] "Generic (PLEG): container finished" podID="ec982344-6200-404b-8beb-9511642c0bff" containerID="2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d" exitCode=0
Jul 29 20:59:40.175556 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.175051    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl"
Jul 29 20:59:40.175556 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.175084    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" event={"ID":"ec982344-6200-404b-8beb-9511642c0bff","Type":"ContainerDied","Data":"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"}
Jul 29 20:59:40.175556 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.175138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cfd5c6dc-gsgdl" event={"ID":"ec982344-6200-404b-8beb-9511642c0bff","Type":"ContainerDied","Data":"19f340a39c62301ff05f0322c288b76516513be476c170dd3a9cdfdb8dfdccd3"}
Jul 29 20:59:40.175556 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.175159    2581 scope.go:117] "RemoveContainer" containerID="2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"
Jul 29 20:59:40.183883 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.183865    2581 scope.go:117] "RemoveContainer" containerID="2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"
Jul 29 20:59:40.184145 ip-10-0-135-229 kubenswrapper[2581]: E0729 20:59:40.184127    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d\": container with ID starting with 2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d not found: ID does not exist" containerID="2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"
Jul 29 20:59:40.184199 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.184154    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d"} err="failed to get container status \"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d\": rpc error: code = NotFound desc = could not find container \"2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d\": container with ID starting with 2919d04e0a5e3e4d237060fd96f8e48235c9bc8152ac9aced8c463a9eb5d134d not found: ID does not exist"
Jul 29 20:59:40.195651 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.195523    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:40.197642 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.197621    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67cfd5c6dc-gsgdl"]
Jul 29 20:59:40.262620 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:40.262580    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec982344-6200-404b-8beb-9511642c0bff" path="/var/lib/kubelet/pods/ec982344-6200-404b-8beb-9511642c0bff/volumes"
Jul 29 20:59:52.343769 ip-10-0-135-229 kubenswrapper[2581]: I0729 20:59:52.343731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:00.411100 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:00.411062    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:00.519272 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:00.519235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:02.106961 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.106925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:02.145405 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.145345    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"]
Jul 29 21:00:02.145746 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.145733    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec982344-6200-404b-8beb-9511642c0bff" containerName="manager"
Jul 29 21:00:02.145805 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.145747    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec982344-6200-404b-8beb-9511642c0bff" containerName="manager"
Jul 29 21:00:02.145841 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.145810    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec982344-6200-404b-8beb-9511642c0bff" containerName="manager"
Jul 29 21:00:02.150518 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.150495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.153460 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.153437    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jv5fn\""
Jul 29 21:00:02.153579 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.153446    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 21:00:02.160143 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.160112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"]
Jul 29 21:00:02.232181 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.232149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-openshift-service-ca\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.232346 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.232274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-maas-api-tls\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.232346 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.232299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2xqb\" (UniqueName: \"kubernetes.io/projected/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-kube-api-access-z2xqb\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.333831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.333785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-maas-api-tls\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.333831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.333829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2xqb\" (UniqueName: \"kubernetes.io/projected/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-kube-api-access-z2xqb\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.334118 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.333872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-openshift-service-ca\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.334621 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.334600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-openshift-service-ca\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.336464 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.336440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-maas-api-tls\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.342261 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.342238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2xqb\" (UniqueName: \"kubernetes.io/projected/bf232bb7-f5b5-48c7-b913-6f6ef0a8e115-kube-api-access-z2xqb\") pod \"maas-api-5bc489dd8-lms2j\" (UID: \"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115\") " pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.463247 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.463159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:02.473591 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.473555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:02.612600 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.612573    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"]
Jul 29 21:00:02.615924 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:00:02.615888    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf232bb7_f5b5_48c7_b913_6f6ef0a8e115.slice/crio-b3fcda964afb111212d5063e6c574b8b58229396e7d4004efd017287aac445b4 WatchSource:0}: Error finding container b3fcda964afb111212d5063e6c574b8b58229396e7d4004efd017287aac445b4: Status 404 returned error can't find the container with id b3fcda964afb111212d5063e6c574b8b58229396e7d4004efd017287aac445b4
Jul 29 21:00:02.664767 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.664730    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:02.705513 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.705477    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"]
Jul 29 21:00:02.709020 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.708996    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lzqhr"]
Jul 29 21:00:02.709153 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.709112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.711122 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.711101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.711401 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.711372    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 21:00:02.711505 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.711492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7jhwn\""
Jul 29 21:00:02.718000 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.717972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"]
Jul 29 21:00:02.721894 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.721863    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lzqhr"]
Jul 29 21:00:02.838787 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.838755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/403f5856-237a-4378-8de5-3d66da23c397-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.839146 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.838801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ef1b538-304f-47b1-a41b-8f9297da9cfd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.839146 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.838828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qfd\" (UniqueName: \"kubernetes.io/projected/403f5856-237a-4378-8de5-3d66da23c397-kube-api-access-n7qfd\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.839146 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.838910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmb7h\" (UniqueName: \"kubernetes.io/projected/9ef1b538-304f-47b1-a41b-8f9297da9cfd-kube-api-access-hmb7h\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.892985 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.892950    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:02.940022 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.939992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/403f5856-237a-4378-8de5-3d66da23c397-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.940228 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.940034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ef1b538-304f-47b1-a41b-8f9297da9cfd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.940228 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.940058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qfd\" (UniqueName: \"kubernetes.io/projected/403f5856-237a-4378-8de5-3d66da23c397-kube-api-access-n7qfd\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.940228 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.940219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmb7h\" (UniqueName: \"kubernetes.io/projected/9ef1b538-304f-47b1-a41b-8f9297da9cfd-kube-api-access-hmb7h\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.940828 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.940805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/403f5856-237a-4378-8de5-3d66da23c397-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.940926 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.940812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ef1b538-304f-47b1-a41b-8f9297da9cfd-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:02.948459 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.948431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qfd\" (UniqueName: \"kubernetes.io/projected/403f5856-237a-4378-8de5-3d66da23c397-kube-api-access-n7qfd\") pod \"payload-processing-5cc6cd444b-lzqhr\" (UID: \"403f5856-237a-4378-8de5-3d66da23c397\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:02.948563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:02.948431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmb7h\" (UniqueName: \"kubernetes.io/projected/9ef1b538-304f-47b1-a41b-8f9297da9cfd-kube-api-access-hmb7h\") pod \"payload-pre-processing-687dbb6cf6-mbw4b\" (UID: \"9ef1b538-304f-47b1-a41b-8f9297da9cfd\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:03.023328 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.023289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:03.030042 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.030017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:03.175245 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.175213    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"]
Jul 29 21:00:03.180024 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:00:03.179992    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef1b538_304f_47b1_a41b_8f9297da9cfd.slice/crio-7e6a5f885575938331e7d61043f65fdf27b42a0911816da750b034131bcb6299 WatchSource:0}: Error finding container 7e6a5f885575938331e7d61043f65fdf27b42a0911816da750b034131bcb6299: Status 404 returned error can't find the container with id 7e6a5f885575938331e7d61043f65fdf27b42a0911816da750b034131bcb6299
Jul 29 21:00:03.197598 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.197562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lzqhr"]
Jul 29 21:00:03.200792 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:00:03.200753    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403f5856_237a_4378_8de5_3d66da23c397.slice/crio-8eb58f17ec7fa475907281a6e2f761a6eaaf4236bf79a5f54bffd8e0d3fda671 WatchSource:0}: Error finding container 8eb58f17ec7fa475907281a6e2f761a6eaaf4236bf79a5f54bffd8e0d3fda671: Status 404 returned error can't find the container with id 8eb58f17ec7fa475907281a6e2f761a6eaaf4236bf79a5f54bffd8e0d3fda671
Jul 29 21:00:03.287280 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.287191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr" event={"ID":"403f5856-237a-4378-8de5-3d66da23c397","Type":"ContainerStarted","Data":"8eb58f17ec7fa475907281a6e2f761a6eaaf4236bf79a5f54bffd8e0d3fda671"}
Jul 29 21:00:03.288508 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.288482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b" event={"ID":"9ef1b538-304f-47b1-a41b-8f9297da9cfd","Type":"ContainerStarted","Data":"7e6a5f885575938331e7d61043f65fdf27b42a0911816da750b034131bcb6299"}
Jul 29 21:00:03.289634 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:03.289610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j" event={"ID":"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115","Type":"ContainerStarted","Data":"b3fcda964afb111212d5063e6c574b8b58229396e7d4004efd017287aac445b4"}
Jul 29 21:00:06.312838 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.311529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j" event={"ID":"bf232bb7-f5b5-48c7-b913-6f6ef0a8e115","Type":"ContainerStarted","Data":"304efe943362a2acae856d2b3f4c63354a01a655d547e7b9fa93d3f41de1c502"}
Jul 29 21:00:06.312838 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.312719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:06.320143 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.320110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr" event={"ID":"403f5856-237a-4378-8de5-3d66da23c397","Type":"ContainerStarted","Data":"f00ade089c1cd218113f9eec0e4451c879a0ab6e6a8377f6590ff1e5d723762b"}
Jul 29 21:00:06.320301 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.320172    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:06.321929 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.321901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b" event={"ID":"9ef1b538-304f-47b1-a41b-8f9297da9cfd","Type":"ContainerStarted","Data":"21cd1131aa0d98f4efc533e7ca4a1cf06de1abf8ab34572e8e2809ff08205597"}
Jul 29 21:00:06.322078 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.321988    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:06.333625 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.333568    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j" podStartSLOduration=0.889436308 podStartE2EDuration="4.333549467s" totalImagesPullingTime="3.444113159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:06.331114657 +0000 UTC m=+756.580352172" watchObservedRunningTime="2026-07-29 21:00:06.333549467 +0000 UTC m=+756.582786982"
Jul 29 21:00:06.348238 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.348179    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b" podStartSLOduration=1.435962082 podStartE2EDuration="4.348161355s" totalImagesPullingTime="2.912199273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:06.346120187 +0000 UTC m=+756.595357703" watchObservedRunningTime="2026-07-29 21:00:06.348161355 +0000 UTC m=+756.597398869"
Jul 29 21:00:06.360803 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:06.360742    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr" podStartSLOduration=1.466634035 podStartE2EDuration="4.360721834s" totalImagesPullingTime="2.894087799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:06.360319996 +0000 UTC m=+756.609557511" watchObservedRunningTime="2026-07-29 21:00:06.360721834 +0000 UTC m=+756.609959348"
Jul 29 21:00:13.338502 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:13.338468    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bc489dd8-lms2j"
Jul 29 21:00:14.281856 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:14.281824    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:14.384317 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:14.384282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:17.330435 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:17.330405    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-lzqhr"
Jul 29 21:00:17.330816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:17.330587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-mbw4b"
Jul 29 21:00:26.550362 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.550326    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"]
Jul 29 21:00:26.552796 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.552779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.555853 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.555787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xm4qm\""
Jul 29 21:00:26.555853 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.555804    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 21:00:26.555853 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.555793    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 21:00:26.556100 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.555852    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 21:00:26.564571 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.564529    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"]
Jul 29 21:00:26.590983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.590955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:26.657768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.657729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s6hj\" (UniqueName: \"kubernetes.io/projected/34331d85-7713-4627-9aaa-f1d515907336-kube-api-access-5s6hj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.657768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.657767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.658013 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.657786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.658013 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.657867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.658013 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.657967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.658162 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.658040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34331d85-7713-4627-9aaa-f1d515907336-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759443 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759443 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759443 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34331d85-7713-4627-9aaa-f1d515907336-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759443 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s6hj\" (UniqueName: \"kubernetes.io/projected/34331d85-7713-4627-9aaa-f1d515907336-kube-api-access-5s6hj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759673 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759673 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.759778 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.759708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.761747 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.761721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34331d85-7713-4627-9aaa-f1d515907336-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.761875 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.761858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34331d85-7713-4627-9aaa-f1d515907336-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.766929 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.766909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s6hj\" (UniqueName: \"kubernetes.io/projected/34331d85-7713-4627-9aaa-f1d515907336-kube-api-access-5s6hj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f\" (UID: \"34331d85-7713-4627-9aaa-f1d515907336\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:26.867745 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:26.867651    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:27.003446 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:27.003420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"]
Jul 29 21:00:27.005512 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:00:27.005481    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34331d85_7713_4627_9aaa_f1d515907336.slice/crio-d14bf1684eb1148fbf8fb5a4725e84a6b860ba5796a5f2285149121acf849ac2 WatchSource:0}: Error finding container d14bf1684eb1148fbf8fb5a4725e84a6b860ba5796a5f2285149121acf849ac2: Status 404 returned error can't find the container with id d14bf1684eb1148fbf8fb5a4725e84a6b860ba5796a5f2285149121acf849ac2
Jul 29 21:00:27.428694 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:27.428661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f" event={"ID":"34331d85-7713-4627-9aaa-f1d515907336","Type":"ContainerStarted","Data":"d14bf1684eb1148fbf8fb5a4725e84a6b860ba5796a5f2285149121acf849ac2"}
Jul 29 21:00:35.475226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:35.475187    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f" event={"ID":"34331d85-7713-4627-9aaa-f1d515907336","Type":"ContainerStarted","Data":"fe18f180afd56762a461c028d7dd928d24a4e603f5751dc50ed2c933a1f97f12"}
Jul 29 21:00:35.494763 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:35.494716    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f" podStartSLOduration=1.892589964 podStartE2EDuration="9.494699922s" totalImagesPullingTime="7.602109958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:35.492624581 +0000 UTC m=+785.741862107" watchObservedRunningTime="2026-07-29 21:00:35.494699922 +0000 UTC m=+785.743937437"
Jul 29 21:00:36.867797 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:36.867759    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:36.868218 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:36.867847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:36.880870 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:36.880837    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:37.497663 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:37.497635    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f"
Jul 29 21:00:46.067176 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:46.067138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:51.550736 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.550613    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"]
Jul 29 21:00:51.555648 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.555629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.557680 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.557659    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 21:00:51.562065 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.562043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"]
Jul 29 21:00:51.642712 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.642668    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:51.693795 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d30376-18e4-4dbb-adcb-9cac2d590373-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.693795 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.693983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4wq\" (UniqueName: \"kubernetes.io/projected/f1d30376-18e4-4dbb-adcb-9cac2d590373-kube-api-access-fc4wq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.693983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.693983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.693983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.693981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795243 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4wq\" (UniqueName: \"kubernetes.io/projected/f1d30376-18e4-4dbb-adcb-9cac2d590373-kube-api-access-fc4wq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795243 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795503 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795275    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795503 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795624 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d30376-18e4-4dbb-adcb-9cac2d590373-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795678 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795725 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795819 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.795917 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.795903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.797670 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.797645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d30376-18e4-4dbb-adcb-9cac2d590373-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.798005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.797985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d30376-18e4-4dbb-adcb-9cac2d590373-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.803060 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.802992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4wq\" (UniqueName: \"kubernetes.io/projected/f1d30376-18e4-4dbb-adcb-9cac2d590373-kube-api-access-fc4wq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9jjwv\" (UID: \"f1d30376-18e4-4dbb-adcb-9cac2d590373\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:51.867530 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:51.867494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:00:52.000741 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:52.000718    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"]
Jul 29 21:00:52.002857 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:00:52.002828    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d30376_18e4_4dbb_adcb_9cac2d590373.slice/crio-c9a64156a764832d879e934111327b4e414c5397a2360f9fde4356877ff03080 WatchSource:0}: Error finding container c9a64156a764832d879e934111327b4e414c5397a2360f9fde4356877ff03080: Status 404 returned error can't find the container with id c9a64156a764832d879e934111327b4e414c5397a2360f9fde4356877ff03080
Jul 29 21:00:52.557642 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:52.557604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv" event={"ID":"f1d30376-18e4-4dbb-adcb-9cac2d590373","Type":"ContainerStarted","Data":"c9a64156a764832d879e934111327b4e414c5397a2360f9fde4356877ff03080"}
Jul 29 21:00:54.568838 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:54.568794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv" event={"ID":"f1d30376-18e4-4dbb-adcb-9cac2d590373","Type":"ContainerStarted","Data":"14d48088186bd8bc8e62981f5f0b94d96c6856ad61bf1a4073b2005a861670d7"}
Jul 29 21:00:54.590254 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:00:54.590198    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv" podStartSLOduration=1.225248683 podStartE2EDuration="3.590181768s" totalImagesPullingTime="2.364933085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:54.588276858 +0000 UTC m=+804.837514375" watchObservedRunningTime="2026-07-29 21:00:54.590181768 +0000 UTC m=+804.839419282"
Jul 29 21:01:01.867740 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:01.867707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:01:01.868211 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:01.867786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:01:01.880076 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:01.880045    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:01:02.616307 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:02.616281    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9jjwv"
Jul 29 21:01:14.054524 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:14.054487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:20.945098 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:20.945057    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"]
Jul 29 21:01:20.950717 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:20.950694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:20.952884 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:20.952857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 21:01:20.960789 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:20.960767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"]
Jul 29 21:01:20.974334 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:20.974299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:21.061348 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.061348 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.061618 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqrv7\" (UniqueName: \"kubernetes.io/projected/cbfd2994-2cac-497d-a659-4fb3642d64ee-kube-api-access-xqrv7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.061618 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.061618 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.061618 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.061606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbfd2994-2cac-497d-a659-4fb3642d64ee-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162573 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbfd2994-2cac-497d-a659-4fb3642d64ee-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162766 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162766 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162766 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqrv7\" (UniqueName: \"kubernetes.io/projected/cbfd2994-2cac-497d-a659-4fb3642d64ee-kube-api-access-xqrv7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162766 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.162973 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.162801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.163097 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.163070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.163178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.163111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.163178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.163150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.165092 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.165073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbfd2994-2cac-497d-a659-4fb3642d64ee-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.165239 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.165222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbfd2994-2cac-497d-a659-4fb3642d64ee-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.175291 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.175270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqrv7\" (UniqueName: \"kubernetes.io/projected/cbfd2994-2cac-497d-a659-4fb3642d64ee-kube-api-access-xqrv7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-28555\" (UID: \"cbfd2994-2cac-497d-a659-4fb3642d64ee\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.262172 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.262122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:21.397801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.397774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"]
Jul 29 21:01:21.400167 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:01:21.400142    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfd2994_2cac_497d_a659_4fb3642d64ee.slice/crio-2cb3f8ab7a5c491be2eaac6a3528d3d7b87ae950b986c9fa1ccdd28c67cfd8a4 WatchSource:0}: Error finding container 2cb3f8ab7a5c491be2eaac6a3528d3d7b87ae950b986c9fa1ccdd28c67cfd8a4: Status 404 returned error can't find the container with id 2cb3f8ab7a5c491be2eaac6a3528d3d7b87ae950b986c9fa1ccdd28c67cfd8a4
Jul 29 21:01:21.700617 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.700582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555" event={"ID":"cbfd2994-2cac-497d-a659-4fb3642d64ee","Type":"ContainerStarted","Data":"fc77702c2e00c313852c4e0e45db30755ab3482e8ca0733173970119a3d25e99"}
Jul 29 21:01:21.700775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.700632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555" event={"ID":"cbfd2994-2cac-497d-a659-4fb3642d64ee","Type":"ContainerStarted","Data":"2cb3f8ab7a5c491be2eaac6a3528d3d7b87ae950b986c9fa1ccdd28c67cfd8a4"}
Jul 29 21:01:21.717749 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:21.717703    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555" podStartSLOduration=1.5506878720000001 podStartE2EDuration="1.717688182s" totalImagesPullingTime="167.00031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:01:21.716210772 +0000 UTC m=+831.965448297" watchObservedRunningTime="2026-07-29 21:01:21.717688182 +0000 UTC m=+831.966925727"
Jul 29 21:01:26.953784 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:26.953737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:31.262229 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:31.262196    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:31.262714 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:31.262275    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:31.274956 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:31.274935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:31.762800 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:31.762775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-28555"
Jul 29 21:01:54.512868 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:01:54.512837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:03.398033 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:03.398000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:07.253221 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:07.253187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:30.607788 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:30.607635    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:02:30.612688 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:30.609784    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:02:30.782571 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:30.782537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:30.884671 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:30.884581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:31.913956 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:31.913915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:32.016264 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:02:32.016230    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:03:09.069592 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:09.069554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:03:17.401180 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.401143    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 21:03:17.401777 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.401372    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-576968bbcc-6jfhm" podUID="f2ae6ec9-e765-4ec1-b102-d3a761793388" containerName="manager" containerID="cri-o://923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf" gracePeriod=30
Jul 29 21:03:17.664164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.664101    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 21:03:17.786798 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.786757    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdhj8\" (UniqueName: \"kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8\") pod \"f2ae6ec9-e765-4ec1-b102-d3a761793388\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") "
Jul 29 21:03:17.786976 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.786860    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") pod \"f2ae6ec9-e765-4ec1-b102-d3a761793388\" (UID: \"f2ae6ec9-e765-4ec1-b102-d3a761793388\") "
Jul 29 21:03:17.789174 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.789139    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8" (OuterVolumeSpecName: "kube-api-access-bdhj8") pod "f2ae6ec9-e765-4ec1-b102-d3a761793388" (UID: "f2ae6ec9-e765-4ec1-b102-d3a761793388"). InnerVolumeSpecName "kube-api-access-bdhj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:03:17.789313 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.789186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f2ae6ec9-e765-4ec1-b102-d3a761793388" (UID: "f2ae6ec9-e765-4ec1-b102-d3a761793388"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:03:17.888626 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.888590    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2ae6ec9-e765-4ec1-b102-d3a761793388-webhook-cert\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:03:17.888626 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:17.888619    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdhj8\" (UniqueName: \"kubernetes.io/projected/f2ae6ec9-e765-4ec1-b102-d3a761793388-kube-api-access-bdhj8\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:03:18.283137 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.283105    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2ae6ec9-e765-4ec1-b102-d3a761793388" containerID="923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf" exitCode=0
Jul 29 21:03:18.283262 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.283146    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-6jfhm"
Jul 29 21:03:18.283262 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.283185    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-6jfhm" event={"ID":"f2ae6ec9-e765-4ec1-b102-d3a761793388","Type":"ContainerDied","Data":"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"}
Jul 29 21:03:18.283262 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.283231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-6jfhm" event={"ID":"f2ae6ec9-e765-4ec1-b102-d3a761793388","Type":"ContainerDied","Data":"634e938432e99f5ac4ce1a533f0c580c14e3f845fbcabf9bccbfd31df909fc3f"}
Jul 29 21:03:18.283262 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.283257    2581 scope.go:117] "RemoveContainer" containerID="923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"
Jul 29 21:03:18.291946 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.291917    2581 scope.go:117] "RemoveContainer" containerID="923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"
Jul 29 21:03:18.292224 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:03:18.292190    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf\": container with ID starting with 923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf not found: ID does not exist" containerID="923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"
Jul 29 21:03:18.292224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.292217    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf"} err="failed to get container status \"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf\": rpc error: code = NotFound desc = could not find container \"923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf\": container with ID starting with 923cfda3145409dbcd64a501aaad6cb6b477f7d5cfaa5c2a7bd56cc651edf4cf not found: ID does not exist"
Jul 29 21:03:18.300119 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.300092    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 21:03:18.304413 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:18.304377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-576968bbcc-6jfhm"]
Jul 29 21:03:20.263226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:03:20.263192    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2ae6ec9-e765-4ec1-b102-d3a761793388" path="/var/lib/kubelet/pods/f2ae6ec9-e765-4ec1-b102-d3a761793388/volumes"
Jul 29 21:04:19.067455 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.067421    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:19.067901 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.067829    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2ae6ec9-e765-4ec1-b102-d3a761793388" containerName="manager"
Jul 29 21:04:19.067901 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.067840    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2ae6ec9-e765-4ec1-b102-d3a761793388" containerName="manager"
Jul 29 21:04:19.068005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.067913    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2ae6ec9-e765-4ec1-b102-d3a761793388" containerName="manager"
Jul 29 21:04:19.071243 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.071225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.073573 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.073552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 21:04:19.074457 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.074440    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 21:04:19.074560 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.074456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bt8mv\""
Jul 29 21:04:19.080893 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.080869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:19.107611 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.107580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.107799 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.107639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm49q\" (UniqueName: \"kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.208713 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.208667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.208914 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.208744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm49q\" (UniqueName: \"kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.211220 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.211197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.217073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.217048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm49q\" (UniqueName: \"kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q\") pod \"maas-controller-576968bbcc-5l79b\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") " pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.383033 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.382937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:19.526635 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.526608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:19.528430 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:04:19.528374    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbe3ab3_5711_478f_9ff8_8df32a53ed62.slice/crio-7208767825b60e121f64aed87ace5442f67f5b419f39cccf6890b62043851f15 WatchSource:0}: Error finding container 7208767825b60e121f64aed87ace5442f67f5b419f39cccf6890b62043851f15: Status 404 returned error can't find the container with id 7208767825b60e121f64aed87ace5442f67f5b419f39cccf6890b62043851f15
Jul 29 21:04:19.529753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.529734    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:04:19.578634 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:19.578598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-5l79b" event={"ID":"0fbe3ab3-5711-478f-9ff8-8df32a53ed62","Type":"ContainerStarted","Data":"7208767825b60e121f64aed87ace5442f67f5b419f39cccf6890b62043851f15"}
Jul 29 21:04:20.585620 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:20.585588    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-5l79b" event={"ID":"0fbe3ab3-5711-478f-9ff8-8df32a53ed62","Type":"ContainerStarted","Data":"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"}
Jul 29 21:04:20.586084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:20.585716    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:20.603816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:20.603757    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-576968bbcc-5l79b" podStartSLOduration=1.187284716 podStartE2EDuration="1.603739315s" totalImagesPullingTime="416.454599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:20.60200468 +0000 UTC m=+1010.851242196" watchObservedRunningTime="2026-07-29 21:04:20.603739315 +0000 UTC m=+1010.852976830"
Jul 29 21:04:31.592821 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:31.592790    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:36.124567 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.124512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:04:36.227424 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.227329    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:36.227686 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.227653    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-576968bbcc-5l79b" podUID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" containerName="manager" containerID="cri-o://019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392" gracePeriod=30
Jul 29 21:04:36.479890 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.479859    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:36.585776 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.585741    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert\") pod \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") "
Jul 29 21:04:36.585953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.585873    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm49q\" (UniqueName: \"kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q\") pod \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\" (UID: \"0fbe3ab3-5711-478f-9ff8-8df32a53ed62\") "
Jul 29 21:04:36.588089 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.588055    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0fbe3ab3-5711-478f-9ff8-8df32a53ed62" (UID: "0fbe3ab3-5711-478f-9ff8-8df32a53ed62"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:04:36.588180 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.588082    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q" (OuterVolumeSpecName: "kube-api-access-xm49q") pod "0fbe3ab3-5711-478f-9ff8-8df32a53ed62" (UID: "0fbe3ab3-5711-478f-9ff8-8df32a53ed62"). InnerVolumeSpecName "kube-api-access-xm49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:04:36.667173 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.667079    2581 generic.go:356] "Generic (PLEG): container finished" podID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" containerID="019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392" exitCode=0
Jul 29 21:04:36.667173 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.667125    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-5l79b"
Jul 29 21:04:36.667173 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.667164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-5l79b" event={"ID":"0fbe3ab3-5711-478f-9ff8-8df32a53ed62","Type":"ContainerDied","Data":"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"}
Jul 29 21:04:36.667508 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.667216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-5l79b" event={"ID":"0fbe3ab3-5711-478f-9ff8-8df32a53ed62","Type":"ContainerDied","Data":"7208767825b60e121f64aed87ace5442f67f5b419f39cccf6890b62043851f15"}
Jul 29 21:04:36.667508 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.667240    2581 scope.go:117] "RemoveContainer" containerID="019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"
Jul 29 21:04:36.676758 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.676738    2581 scope.go:117] "RemoveContainer" containerID="019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"
Jul 29 21:04:36.677036 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:04:36.677019    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392\": container with ID starting with 019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392 not found: ID does not exist" containerID="019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"
Jul 29 21:04:36.677078 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.677045    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392"} err="failed to get container status \"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392\": rpc error: code = NotFound desc = could not find container \"019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392\": container with ID starting with 019ae4284b0a43ff356fe8c12bc5ddf518f8ab4b15b41ebcb2686565b9f1f392 not found: ID does not exist"
Jul 29 21:04:36.686929 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.686900    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-webhook-cert\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:04:36.686929 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.686930    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm49q\" (UniqueName: \"kubernetes.io/projected/0fbe3ab3-5711-478f-9ff8-8df32a53ed62-kube-api-access-xm49q\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:04:36.688205 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.688182    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:36.690903 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:36.690868    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-576968bbcc-5l79b"]
Jul 29 21:04:37.327282 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.327254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:04:37.448803 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.448768    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-576968bbcc-pgxwx"]
Jul 29 21:04:37.449249 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.449230    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" containerName="manager"
Jul 29 21:04:37.449249 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.449253    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" containerName="manager"
Jul 29 21:04:37.449358 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.449323    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" containerName="manager"
Jul 29 21:04:37.453545 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.453526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.456327 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.456308    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bt8mv\""
Jul 29 21:04:37.456563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.456349    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 21:04:37.456563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.456350    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 21:04:37.460659 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.460638    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-pgxwx"]
Jul 29 21:04:37.596891 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.596797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svflt\" (UniqueName: \"kubernetes.io/projected/118d3809-c82a-47df-bb10-797189ae0102-kube-api-access-svflt\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.596891 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.596880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118d3809-c82a-47df-bb10-797189ae0102-webhook-cert\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.697523 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.697481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118d3809-c82a-47df-bb10-797189ae0102-webhook-cert\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.697717 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.697599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svflt\" (UniqueName: \"kubernetes.io/projected/118d3809-c82a-47df-bb10-797189ae0102-kube-api-access-svflt\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.700080 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.700060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118d3809-c82a-47df-bb10-797189ae0102-webhook-cert\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.706918 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.706893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svflt\" (UniqueName: \"kubernetes.io/projected/118d3809-c82a-47df-bb10-797189ae0102-kube-api-access-svflt\") pod \"maas-controller-576968bbcc-pgxwx\" (UID: \"118d3809-c82a-47df-bb10-797189ae0102\") " pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.766131 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.766091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:37.936174 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:37.935962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-576968bbcc-pgxwx"]
Jul 29 21:04:37.938728 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:04:37.938701    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118d3809_c82a_47df_bb10_797189ae0102.slice/crio-f90e60506a373a428c9d7c8fe3d392c4f916f56b79e38d22674030c5b2e82f3c WatchSource:0}: Error finding container f90e60506a373a428c9d7c8fe3d392c4f916f56b79e38d22674030c5b2e82f3c: Status 404 returned error can't find the container with id f90e60506a373a428c9d7c8fe3d392c4f916f56b79e38d22674030c5b2e82f3c
Jul 29 21:04:38.263663 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:38.263629    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fbe3ab3-5711-478f-9ff8-8df32a53ed62" path="/var/lib/kubelet/pods/0fbe3ab3-5711-478f-9ff8-8df32a53ed62/volumes"
Jul 29 21:04:38.680267 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:38.680165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-pgxwx" event={"ID":"118d3809-c82a-47df-bb10-797189ae0102","Type":"ContainerStarted","Data":"a66b8c2235669e3797a7996d992ee7f01c136f2c4f71cb0b4cd74831d7d2086c"}
Jul 29 21:04:38.680744 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:38.680273    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:04:38.680744 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:38.680297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-576968bbcc-pgxwx" event={"ID":"118d3809-c82a-47df-bb10-797189ae0102","Type":"ContainerStarted","Data":"f90e60506a373a428c9d7c8fe3d392c4f916f56b79e38d22674030c5b2e82f3c"}
Jul 29 21:04:38.697846 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:38.697787    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-576968bbcc-pgxwx" podStartSLOduration=1.2920130300000001 podStartE2EDuration="1.697768215s" totalImagesPullingTime="405.755185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:38.696660611 +0000 UTC m=+1028.945898149" watchObservedRunningTime="2026-07-29 21:04:38.697768215 +0000 UTC m=+1028.947005728"
Jul 29 21:04:49.689091 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:04:49.689062    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-576968bbcc-pgxwx"
Jul 29 21:05:11.554576 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:11.554534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:11.659392 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:11.659345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:11.777803 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:11.777770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:17.583188 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:17.583147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:17.690479 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:17.690434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:17.783449 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:17.783412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:17.914573 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:17.914479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:23.566341 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:23.566261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:23.665252 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:23.665217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:49.328916 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:49.328883    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:57.631511 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:57.631476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:05:57.732656 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:05:57.732626    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:07:30.639245 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:07:30.639141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:07:30.643311 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:07:30.641768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:08:21.205350 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:08:21.205277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:58.293145 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:11:58.293028    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:58.378584 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:11:58.378553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:12:30.678711 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:12:30.678601    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:12:30.682738 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:12:30.681541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:13:18.901158 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:13:18.901117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:13:35.101370 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:13:35.101330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:13:40.874963 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:13:40.874928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:14:13.448428 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:14:13.448394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:14:14.986199 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:14:14.986165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:30.710262 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:17:30.710149    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:17:30.715061 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:17:30.715041    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:17:36.521113 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:17:36.521077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:58.586171 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:17:58.586131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.174986 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:02.174951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.350693 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:02.350651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.661942 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:02.661914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.689490 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:08.689460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.790159 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:08.790126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.297760 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:09.297713    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.396878 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:09.396838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.824654 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:09.824618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:11.408424 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:11.408363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:11.495217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:11.495185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.561357 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.561310    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:12.565155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.565128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.571432 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.571407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dbfaf7e5-serving-cert\""
Jul 29 21:18:12.583158 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.583130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.583546 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.583518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.583697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.583678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4h77\" (UniqueName: \"kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.586399 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.586364    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:12.656724 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.656691    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.684710 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.684673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.684889 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.684744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4h77\" (UniqueName: \"kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.684889 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.684801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.685483 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.685454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.687564 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.687542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.696650 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.696620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4h77\" (UniqueName: \"kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77\") pod \"maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:12.839187 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.839085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.875362 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:12.875327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:13.046797 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.046773    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:13.049402 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:18:13.049354    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6783ea24_e616_496c_9868_403c4605cc19.slice/crio-8460ff54cc2d07563a9914eed25d6e0e3e2ed3ddf4671d5f5d7828007c2de312 WatchSource:0}: Error finding container 8460ff54cc2d07563a9914eed25d6e0e3e2ed3ddf4671d5f5d7828007c2de312: Status 404 returned error can't find the container with id 8460ff54cc2d07563a9914eed25d6e0e3e2ed3ddf4671d5f5d7828007c2de312
Jul 29 21:18:13.050831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.050814    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:18:13.501905 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.501873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:13.614469 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.614439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:13.660759 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.660723    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:13.667436 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.667408    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.675073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.675047    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-dbfaf7e5\""
Jul 29 21:18:13.686633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.686603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:13.695879 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.695845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvtj8\" (UniqueName: \"kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.696062 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.695914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.748443 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.748395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" event={"ID":"6783ea24-e616-496c-9868-403c4605cc19","Type":"ContainerStarted","Data":"8460ff54cc2d07563a9914eed25d6e0e3e2ed3ddf4671d5f5d7828007c2de312"}
Jul 29 21:18:13.797274 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.797183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvtj8\" (UniqueName: \"kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.797529 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.797377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.798132 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.798107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.818973 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.818945    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvtj8\" (UniqueName: \"kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8\") pod \"payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:13.979650 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:13.979621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:14.350672 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.350646    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:14.352365 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:18:14.352333    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bca0777_f599_4390_92bb_189343109049.slice/crio-487d64be1f37e3ff5496a7e23b7c8a75370552cec8429893bd18ebb7da7e2749 WatchSource:0}: Error finding container 487d64be1f37e3ff5496a7e23b7c8a75370552cec8429893bd18ebb7da7e2749: Status 404 returned error can't find the container with id 487d64be1f37e3ff5496a7e23b7c8a75370552cec8429893bd18ebb7da7e2749
Jul 29 21:18:14.755858 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.755821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" event={"ID":"6783ea24-e616-496c-9868-403c4605cc19","Type":"ContainerStarted","Data":"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"}
Jul 29 21:18:14.756322 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.755993    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:14.757352 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.757314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" event={"ID":"6bca0777-f599-4390-92bb-189343109049","Type":"ContainerStarted","Data":"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"}
Jul 29 21:18:14.757504 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.757362    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" event={"ID":"6bca0777-f599-4390-92bb-189343109049","Type":"ContainerStarted","Data":"487d64be1f37e3ff5496a7e23b7c8a75370552cec8429893bd18ebb7da7e2749"}
Jul 29 21:18:14.757504 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.757490    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:14.775486 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.775431    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" podStartSLOduration=1.991989621 podStartE2EDuration="2.77541286s" totalImagesPullingTime="783.423239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:14.772860364 +0000 UTC m=+1845.022097902" watchObservedRunningTime="2026-07-29 21:18:14.77541286 +0000 UTC m=+1845.024650375"
Jul 29 21:18:14.793025 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:14.792974    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" podStartSLOduration=1.7929568470000001 podStartE2EDuration="1.792956847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:14.790821968 +0000 UTC m=+1845.040059484" watchObservedRunningTime="2026-07-29 21:18:14.792956847 +0000 UTC m=+1845.042194361"
Jul 29 21:18:20.768514 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:20.768480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:25.763551 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:25.763501    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" podUID="6bca0777-f599-4390-92bb-189343109049" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.42:9004: connect: connection refused"
Jul 29 21:18:35.762684 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:35.762652    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:43.118843 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.118809    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:43.119251 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.119022    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" podUID="6783ea24-e616-496c-9868-403c4605cc19" containerName="maas-api" containerID="cri-o://b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188" gracePeriod=30
Jul 29 21:18:43.144861 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.144836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:43.179843 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.179809    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:43.180597 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.180066    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" podUID="6bca0777-f599-4390-92bb-189343109049" containerName="payload-pre-processing" containerID="cri-o://37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947" gracePeriod=30
Jul 29 21:18:43.441339 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.441313    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:43.474925 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.474887    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.500951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvtj8\" (UniqueName: \"kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8\") pod \"6bca0777-f599-4390-92bb-189343109049\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") "
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.501011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4h77\" (UniqueName: \"kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77\") pod \"6783ea24-e616-496c-9868-403c4605cc19\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") "
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.501062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume\") pod \"6bca0777-f599-4390-92bb-189343109049\" (UID: \"6bca0777-f599-4390-92bb-189343109049\") "
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.501162    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls\") pod \"6783ea24-e616-496c-9868-403c4605cc19\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") "
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.501215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca\") pod \"6783ea24-e616-496c-9868-403c4605cc19\" (UID: \"6783ea24-e616-496c-9868-403c4605cc19\") "
Jul 29 21:18:43.502772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.502492    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6783ea24-e616-496c-9868-403c4605cc19" (UID: "6783ea24-e616-496c-9868-403c4605cc19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:43.503686 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.503403    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bca0777-f599-4390-92bb-189343109049" (UID: "6bca0777-f599-4390-92bb-189343109049"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:43.508918 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.507668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8" (OuterVolumeSpecName: "kube-api-access-lvtj8") pod "6bca0777-f599-4390-92bb-189343109049" (UID: "6bca0777-f599-4390-92bb-189343109049"). InnerVolumeSpecName "kube-api-access-lvtj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:43.509800 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.509769    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77" (OuterVolumeSpecName: "kube-api-access-w4h77") pod "6783ea24-e616-496c-9868-403c4605cc19" (UID: "6783ea24-e616-496c-9868-403c4605cc19"). InnerVolumeSpecName "kube-api-access-w4h77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:43.510071 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.510040    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6783ea24-e616-496c-9868-403c4605cc19" (UID: "6783ea24-e616-496c-9868-403c4605cc19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:18:43.602813 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.602774    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvtj8\" (UniqueName: \"kubernetes.io/projected/6bca0777-f599-4390-92bb-189343109049-kube-api-access-lvtj8\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:18:43.602813 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.602809    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4h77\" (UniqueName: \"kubernetes.io/projected/6783ea24-e616-496c-9868-403c4605cc19-kube-api-access-w4h77\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:18:43.602813 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.602819    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bca0777-f599-4390-92bb-189343109049-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:18:43.603073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.602829    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6783ea24-e616-496c-9868-403c4605cc19-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:18:43.603073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.602838    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6783ea24-e616-496c-9868-403c4605cc19-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:18:43.906604 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.906566    2581 generic.go:356] "Generic (PLEG): container finished" podID="6bca0777-f599-4390-92bb-189343109049" containerID="37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947" exitCode=0
Jul 29 21:18:43.906812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.906625    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"
Jul 29 21:18:43.906812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.906665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" event={"ID":"6bca0777-f599-4390-92bb-189343109049","Type":"ContainerDied","Data":"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"}
Jul 29 21:18:43.906812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.906758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh" event={"ID":"6bca0777-f599-4390-92bb-189343109049","Type":"ContainerDied","Data":"487d64be1f37e3ff5496a7e23b7c8a75370552cec8429893bd18ebb7da7e2749"}
Jul 29 21:18:43.906812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.906789    2581 scope.go:117] "RemoveContainer" containerID="37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"
Jul 29 21:18:43.908084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.908059    2581 generic.go:356] "Generic (PLEG): container finished" podID="6783ea24-e616-496c-9868-403c4605cc19" containerID="b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188" exitCode=0
Jul 29 21:18:43.908198 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.908113    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"
Jul 29 21:18:43.908198 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.908120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" event={"ID":"6783ea24-e616-496c-9868-403c4605cc19","Type":"ContainerDied","Data":"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"}
Jul 29 21:18:43.908198 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.908166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c" event={"ID":"6783ea24-e616-496c-9868-403c4605cc19","Type":"ContainerDied","Data":"8460ff54cc2d07563a9914eed25d6e0e3e2ed3ddf4671d5f5d7828007c2de312"}
Jul 29 21:18:43.917203 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.917187    2581 scope.go:117] "RemoveContainer" containerID="37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"
Jul 29 21:18:43.917496 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:18:43.917478    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947\": container with ID starting with 37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947 not found: ID does not exist" containerID="37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"
Jul 29 21:18:43.917569 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.917503    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947"} err="failed to get container status \"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947\": rpc error: code = NotFound desc = could not find container \"37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947\": container with ID starting with 37a18d32656d8c2e9e6e97ed140e7dd41bfeed1e121ab93949415455ac0dc947 not found: ID does not exist"
Jul 29 21:18:43.917569 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.917517    2581 scope.go:117] "RemoveContainer" containerID="b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"
Jul 29 21:18:43.925424 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.925407    2581 scope.go:117] "RemoveContainer" containerID="b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"
Jul 29 21:18:43.925725 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:18:43.925708    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188\": container with ID starting with b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188 not found: ID does not exist" containerID="b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"
Jul 29 21:18:43.925775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.925732    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188"} err="failed to get container status \"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188\": rpc error: code = NotFound desc = could not find container \"b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188\": container with ID starting with b3697076d9e807d966acf282be48dd6c6e8450e52e5ac289bdacd18a660fa188 not found: ID does not exist"
Jul 29 21:18:43.934080 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.934059    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:43.937925 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.937904    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dbfaf7e5-545dbcc97b-sv8jh"]
Jul 29 21:18:43.949522 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.949501    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:43.953166 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:43.953144    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dbfaf7e5-694fdd4789-62p4c"]
Jul 29 21:18:44.263460 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:44.263428    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6783ea24-e616-496c-9868-403c4605cc19" path="/var/lib/kubelet/pods/6783ea24-e616-496c-9868-403c4605cc19/volumes"
Jul 29 21:18:44.263816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:44.263808    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bca0777-f599-4390-92bb-189343109049" path="/var/lib/kubelet/pods/6bca0777-f599-4390-92bb-189343109049/volumes"
Jul 29 21:18:45.054793 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:45.054764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:46.715901 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.715871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:46.786922 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.786890    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:18:46.787701 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787676    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6783ea24-e616-496c-9868-403c4605cc19" containerName="maas-api"
Jul 29 21:18:46.787701 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787703    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6783ea24-e616-496c-9868-403c4605cc19" containerName="maas-api"
Jul 29 21:18:46.787904 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787722    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bca0777-f599-4390-92bb-189343109049" containerName="payload-pre-processing"
Jul 29 21:18:46.787904 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787732    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bca0777-f599-4390-92bb-189343109049" containerName="payload-pre-processing"
Jul 29 21:18:46.787904 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787853    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bca0777-f599-4390-92bb-189343109049" containerName="payload-pre-processing"
Jul 29 21:18:46.787904 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.787866    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6783ea24-e616-496c-9868-403c4605cc19" containerName="maas-api"
Jul 29 21:18:46.793014 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.792993    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.795398 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.795356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-9900cabf-serving-cert\""
Jul 29 21:18:46.803903 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.803878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:18:46.819880 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.819848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:46.832191 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.832161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.832330 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.832200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.832429 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.832365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc75l\" (UniqueName: \"kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.933757 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.933721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.933757 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.933757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.934001 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.933845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc75l\" (UniqueName: \"kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.934599 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.934574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.936564 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.936536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.953953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.953927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc75l\" (UniqueName: \"kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l\") pod \"maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:46.981222 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:46.981146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:47.106163 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.106124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:47.252921 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.252884    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:18:47.256237 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:18:47.256206    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb94b7c2_0830_4f0d_a496_65f598ff6677.slice/crio-d64659f3185e60dd8099cd538ed1e90a528019be9b0a5f5c1b0837cb4997d15c WatchSource:0}: Error finding container d64659f3185e60dd8099cd538ed1e90a528019be9b0a5f5c1b0837cb4997d15c: Status 404 returned error can't find the container with id d64659f3185e60dd8099cd538ed1e90a528019be9b0a5f5c1b0837cb4997d15c
Jul 29 21:18:47.389657 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.389613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:47.490067 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.490036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:47.932227 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.932186    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" event={"ID":"fb94b7c2-0830-4f0d-a496-65f598ff6677","Type":"ContainerStarted","Data":"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"}
Jul 29 21:18:47.932881 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.932246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" event={"ID":"fb94b7c2-0830-4f0d-a496-65f598ff6677","Type":"ContainerStarted","Data":"d64659f3185e60dd8099cd538ed1e90a528019be9b0a5f5c1b0837cb4997d15c"}
Jul 29 21:18:47.932881 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:47.932301    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:53.943926 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:53.943846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:18:53.962541 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:53.962494    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" podStartSLOduration=7.502908341 podStartE2EDuration="7.962478827s" totalImagesPullingTime="459.570486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:47.964049296 +0000 UTC m=+1878.213286812" watchObservedRunningTime="2026-07-29 21:18:53.962478827 +0000 UTC m=+1884.211716342"
Jul 29 21:18:55.822920 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:18:55.822882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:17.534070 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.534035    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:19:17.534648 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.534317    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" podUID="fb94b7c2-0830-4f0d-a496-65f598ff6677" containerName="maas-api" containerID="cri-o://8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e" gracePeriod=30
Jul 29 21:19:17.565960 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.565923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:17.653310 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.653277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:17.790581 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.790509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:17.797460 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.797440    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:19:17.863537 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.863510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:17.933999 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.933963    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca\") pod \"fb94b7c2-0830-4f0d-a496-65f598ff6677\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") "
Jul 29 21:19:17.933999 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.934013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls\") pod \"fb94b7c2-0830-4f0d-a496-65f598ff6677\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") "
Jul 29 21:19:17.934258 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.934088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc75l\" (UniqueName: \"kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l\") pod \"fb94b7c2-0830-4f0d-a496-65f598ff6677\" (UID: \"fb94b7c2-0830-4f0d-a496-65f598ff6677\") "
Jul 29 21:19:17.934435 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.934407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb94b7c2-0830-4f0d-a496-65f598ff6677" (UID: "fb94b7c2-0830-4f0d-a496-65f598ff6677"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:19:17.934533 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.934517    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb94b7c2-0830-4f0d-a496-65f598ff6677-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:19:17.937425 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.937220    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb94b7c2-0830-4f0d-a496-65f598ff6677" (UID: "fb94b7c2-0830-4f0d-a496-65f598ff6677"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:19:17.938056 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.938010    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l" (OuterVolumeSpecName: "kube-api-access-zc75l") pod "fb94b7c2-0830-4f0d-a496-65f598ff6677" (UID: "fb94b7c2-0830-4f0d-a496-65f598ff6677"). InnerVolumeSpecName "kube-api-access-zc75l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:19:17.951442 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:17.951414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:18.035178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.035145    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb94b7c2-0830-4f0d-a496-65f598ff6677-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:19:18.035178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.035172    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc75l\" (UniqueName: \"kubernetes.io/projected/fb94b7c2-0830-4f0d-a496-65f598ff6677-kube-api-access-zc75l\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:19:18.095995 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.095905    2581 generic.go:356] "Generic (PLEG): container finished" podID="fb94b7c2-0830-4f0d-a496-65f598ff6677" containerID="8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e" exitCode=0
Jul 29 21:19:18.095995 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.095953    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"
Jul 29 21:19:18.096269 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.095955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" event={"ID":"fb94b7c2-0830-4f0d-a496-65f598ff6677","Type":"ContainerDied","Data":"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"}
Jul 29 21:19:18.096269 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.096082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9" event={"ID":"fb94b7c2-0830-4f0d-a496-65f598ff6677","Type":"ContainerDied","Data":"d64659f3185e60dd8099cd538ed1e90a528019be9b0a5f5c1b0837cb4997d15c"}
Jul 29 21:19:18.096269 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.096112    2581 scope.go:117] "RemoveContainer" containerID="8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"
Jul 29 21:19:18.106016 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.105998    2581 scope.go:117] "RemoveContainer" containerID="8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"
Jul 29 21:19:18.106288 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:19:18.106269    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e\": container with ID starting with 8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e not found: ID does not exist" containerID="8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"
Jul 29 21:19:18.106333 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.106295    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e"} err="failed to get container status \"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e\": rpc error: code = NotFound desc = could not find container \"8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e\": container with ID starting with 8697fca79458adc38e214f2a5d7ebc01e20f1c8df81bb48cbdb1d687e318942e not found: ID does not exist"
Jul 29 21:19:18.117597 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.117571    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:19:18.121695 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.121675    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9900cabf-5f6c5c974d-p8nz9"]
Jul 29 21:19:18.191007 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.190974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:18.263547 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.263520    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb94b7c2-0830-4f0d-a496-65f598ff6677" path="/var/lib/kubelet/pods/fb94b7c2-0830-4f0d-a496-65f598ff6677/volumes"
Jul 29 21:19:18.431159 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.431083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:18.754143 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:18.754111    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:19.085342 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:19.085263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:20.807742 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:20.807707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:20.908164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:20.908132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:21.953616 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.953585    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:19:21.954056 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.954042    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb94b7c2-0830-4f0d-a496-65f598ff6677" containerName="maas-api"
Jul 29 21:19:21.954104 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.954058    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb94b7c2-0830-4f0d-a496-65f598ff6677" containerName="maas-api"
Jul 29 21:19:21.954147 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.954138    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb94b7c2-0830-4f0d-a496-65f598ff6677" containerName="maas-api"
Jul 29 21:19:21.958498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.958482    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:21.960669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.960647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e74c91e4-serving-cert\""
Jul 29 21:19:21.968528 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.968504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:19:21.976798 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.976771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:21.976962 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.976817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpnbn\" (UniqueName: \"kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:21.976962 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:21.976896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.059413 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.059348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:22.077981 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.077948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.078155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.078012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpnbn\" (UniqueName: \"kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.078155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.078072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.078155 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:19:22.078110    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-serving-cert: secret "maas-api-e2e-ait-e74c91e4-serving-cert" not found
Jul 29 21:19:22.078293 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:19:22.078165    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls podName:c53c1cc7-7788-47ec-8e89-64e030091b8c nodeName:}" failed. No retries permitted until 2026-07-29 21:19:22.578148724 +0000 UTC m=+1912.827386218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls") pod "maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" (UID: "c53c1cc7-7788-47ec-8e89-64e030091b8c") : secret "maas-api-e2e-ait-e74c91e4-serving-cert" not found
Jul 29 21:19:22.078733 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.078712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.090321 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.090297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpnbn\" (UniqueName: \"kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.313537 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.313501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:22.561025 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.560989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:22.583739 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.583649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.586627 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.586595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") pod \"maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:22.662946 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.662915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:22.870527 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:22.870440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:23.015596 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:23.015563    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:19:23.019314 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:19:23.019287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53c1cc7_7788_47ec_8e89_64e030091b8c.slice/crio-46e180732cade55574bb223e3085c0ee7496b123ca3a04d81e8960700362d70e WatchSource:0}: Error finding container 46e180732cade55574bb223e3085c0ee7496b123ca3a04d81e8960700362d70e: Status 404 returned error can't find the container with id 46e180732cade55574bb223e3085c0ee7496b123ca3a04d81e8960700362d70e
Jul 29 21:19:23.125284 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:23.125198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" event={"ID":"c53c1cc7-7788-47ec-8e89-64e030091b8c","Type":"ContainerStarted","Data":"46e180732cade55574bb223e3085c0ee7496b123ca3a04d81e8960700362d70e"}
Jul 29 21:19:24.132340 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:24.132305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" event={"ID":"c53c1cc7-7788-47ec-8e89-64e030091b8c","Type":"ContainerStarted","Data":"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"}
Jul 29 21:19:24.132855 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:24.132414    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:24.153155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:24.153099    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" podStartSLOduration=2.733805448 podStartE2EDuration="3.153083068s" totalImagesPullingTime="419.27762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:19:24.15060946 +0000 UTC m=+1914.399846987" watchObservedRunningTime="2026-07-29 21:19:24.153083068 +0000 UTC m=+1914.402320583"
Jul 29 21:19:30.143345 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:30.143319    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:19:55.513142 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:55.513111    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:55.613393 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:19:55.613359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.698648 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.698614    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:20:02.699137 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.698838    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" podUID="c53c1cc7-7788-47ec-8e89-64e030091b8c" containerName="maas-api" containerID="cri-o://49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f" gracePeriod=30
Jul 29 21:20:02.722005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.721982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.802208 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.802177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.860888 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.860855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.928509 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.928474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.955018 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.954961    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:20:02.972845 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.972815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") pod \"c53c1cc7-7788-47ec-8e89-64e030091b8c\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") "
Jul 29 21:20:02.972983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.972961    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca\") pod \"c53c1cc7-7788-47ec-8e89-64e030091b8c\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") "
Jul 29 21:20:02.973083 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.973067    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpnbn\" (UniqueName: \"kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn\") pod \"c53c1cc7-7788-47ec-8e89-64e030091b8c\" (UID: \"c53c1cc7-7788-47ec-8e89-64e030091b8c\") "
Jul 29 21:20:02.973396 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.973354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c53c1cc7-7788-47ec-8e89-64e030091b8c" (UID: "c53c1cc7-7788-47ec-8e89-64e030091b8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:20:02.975877 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.975829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn" (OuterVolumeSpecName: "kube-api-access-lpnbn") pod "c53c1cc7-7788-47ec-8e89-64e030091b8c" (UID: "c53c1cc7-7788-47ec-8e89-64e030091b8c"). InnerVolumeSpecName "kube-api-access-lpnbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:20:02.976139 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:02.976115    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c53c1cc7-7788-47ec-8e89-64e030091b8c" (UID: "c53c1cc7-7788-47ec-8e89-64e030091b8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:20:03.074374 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.074338    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpnbn\" (UniqueName: \"kubernetes.io/projected/c53c1cc7-7788-47ec-8e89-64e030091b8c-kube-api-access-lpnbn\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:03.074374 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.074370    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c53c1cc7-7788-47ec-8e89-64e030091b8c-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:03.074607 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.074407    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53c1cc7-7788-47ec-8e89-64e030091b8c-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:03.327998 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.327962    2581 generic.go:356] "Generic (PLEG): container finished" podID="c53c1cc7-7788-47ec-8e89-64e030091b8c" containerID="49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f" exitCode=0
Jul 29 21:20:03.328217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.328003    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"
Jul 29 21:20:03.328217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.328040    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" event={"ID":"c53c1cc7-7788-47ec-8e89-64e030091b8c","Type":"ContainerDied","Data":"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"}
Jul 29 21:20:03.328217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.328092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw" event={"ID":"c53c1cc7-7788-47ec-8e89-64e030091b8c","Type":"ContainerDied","Data":"46e180732cade55574bb223e3085c0ee7496b123ca3a04d81e8960700362d70e"}
Jul 29 21:20:03.328217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.328119    2581 scope.go:117] "RemoveContainer" containerID="49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"
Jul 29 21:20:03.338217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.338199    2581 scope.go:117] "RemoveContainer" containerID="49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"
Jul 29 21:20:03.338469 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:20:03.338448    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f\": container with ID starting with 49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f not found: ID does not exist" containerID="49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"
Jul 29 21:20:03.338560 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.338481    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f"} err="failed to get container status \"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f\": rpc error: code = NotFound desc = could not find container \"49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f\": container with ID starting with 49695a9ec094bb6fe5e9a2d866df8f92b217d94cb459969b67c66a17b27e1d9f not found: ID does not exist"
Jul 29 21:20:03.350253 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.350224    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:20:03.352300 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:03.352279    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e74c91e4-6b5cbf6f7-xqbtw"]
Jul 29 21:20:04.180033 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:04.180003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:04.269825 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:04.269785    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53c1cc7-7788-47ec-8e89-64e030091b8c" path="/var/lib/kubelet/pods/c53c1cc7-7788-47ec-8e89-64e030091b8c/volumes"
Jul 29 21:20:04.416112 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:04.416068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:04.660111 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:04.660062    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:05.063681 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:05.063647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:05.169253 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:05.169222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:07.116039 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:07.116001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:07.215005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:07.214970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.208245 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.208204    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:08.209501 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.209460    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53c1cc7-7788-47ec-8e89-64e030091b8c" containerName="maas-api"
Jul 29 21:20:08.209501 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.209503    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53c1cc7-7788-47ec-8e89-64e030091b8c" containerName="maas-api"
Jul 29 21:20:08.209663 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.209614    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c53c1cc7-7788-47ec-8e89-64e030091b8c" containerName="maas-api"
Jul 29 21:20:08.214370 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.214351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.216659 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.216635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f1022d1c-serving-cert\""
Jul 29 21:20:08.220717 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.220697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:08.245687 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.245664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.329775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.329735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.329964 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.329816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhrxh\" (UniqueName: \"kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.329964 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.329938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.431164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.431133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.431324 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.431241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.431367 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.431326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhrxh\" (UniqueName: \"kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.431442 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:20:08.431422    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-serving-cert: secret "maas-api-e2e-derive-f1022d1c-serving-cert" not found
Jul 29 21:20:08.431517 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:20:08.431506    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls podName:023ced4c-1b52-4673-8d7d-d4db6ac0eac2 nodeName:}" failed. No retries permitted until 2026-07-29 21:20:08.93148584 +0000 UTC m=+1959.180723339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls") pod "maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" (UID: "023ced4c-1b52-4673-8d7d-d4db6ac0eac2") : secret "maas-api-e2e-derive-f1022d1c-serving-cert" not found
Jul 29 21:20:08.431830 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.431812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.440561 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.440532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhrxh\" (UniqueName: \"kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.559027 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.558995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.846566 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.846489    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.935473 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.935440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.938441 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.938419    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") pod \"maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:08.943704 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:08.943681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:09.001812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.001774    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:09.005462 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.005447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.008097 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.008076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-f1022d1c-dockercfg-2b6jf\""
Jul 29 21:20:09.008434 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.008420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-f1022d1c\""
Jul 29 21:20:09.016197 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.016171    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:09.036724 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.036697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-866vc\" (UniqueName: \"kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.036842 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.036760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.128131 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.128048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:09.138136 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.138109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.138272 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.138251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-866vc\" (UniqueName: \"kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.145401 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.139506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.148414 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.147988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-866vc\" (UniqueName: \"kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc\") pod \"payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") " pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.317481 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.317448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:09.460804 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.460779    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:09.463202 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:20:09.463177    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201b1497_f53a_47ed_94a4_3ce419c298f0.slice/crio-c170c46bef019a64101b98a3e7c748abe8776d685aa8864e868a8c9cfcfd4b34 WatchSource:0}: Error finding container c170c46bef019a64101b98a3e7c748abe8776d685aa8864e868a8c9cfcfd4b34: Status 404 returned error can't find the container with id c170c46bef019a64101b98a3e7c748abe8776d685aa8864e868a8c9cfcfd4b34
Jul 29 21:20:09.485988 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:09.485958    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:09.487190 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:20:09.487165    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023ced4c_1b52_4673_8d7d_d4db6ac0eac2.slice/crio-45c4afa0082d4aec3ea8face0ab4a90096ff266ecf5b304d0682adef8af1b507 WatchSource:0}: Error finding container 45c4afa0082d4aec3ea8face0ab4a90096ff266ecf5b304d0682adef8af1b507: Status 404 returned error can't find the container with id 45c4afa0082d4aec3ea8face0ab4a90096ff266ecf5b304d0682adef8af1b507
Jul 29 21:20:10.364791 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.364760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" event={"ID":"023ced4c-1b52-4673-8d7d-d4db6ac0eac2","Type":"ContainerStarted","Data":"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"}
Jul 29 21:20:10.365230 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.364812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" event={"ID":"023ced4c-1b52-4673-8d7d-d4db6ac0eac2","Type":"ContainerStarted","Data":"45c4afa0082d4aec3ea8face0ab4a90096ff266ecf5b304d0682adef8af1b507"}
Jul 29 21:20:10.365230 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.364880    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:10.366552 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.366528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" event={"ID":"201b1497-f53a-47ed-94a4-3ce419c298f0","Type":"ContainerStarted","Data":"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"}
Jul 29 21:20:10.366669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.366576    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:10.366669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.366599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" event={"ID":"201b1497-f53a-47ed-94a4-3ce419c298f0","Type":"ContainerStarted","Data":"c170c46bef019a64101b98a3e7c748abe8776d685aa8864e868a8c9cfcfd4b34"}
Jul 29 21:20:10.386732 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.386680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" podStartSLOduration=1.683040778 podStartE2EDuration="2.386664905s" totalImagesPullingTime="703.624127ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:20:10.385957447 +0000 UTC m=+1960.635194973" watchObservedRunningTime="2026-07-29 21:20:10.386664905 +0000 UTC m=+1960.635902422"
Jul 29 21:20:10.408080 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:10.408033    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" podStartSLOduration=2.408018223 podStartE2EDuration="2.408018223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:20:10.406417101 +0000 UTC m=+1960.655654635" watchObservedRunningTime="2026-07-29 21:20:10.408018223 +0000 UTC m=+1960.657255738"
Jul 29 21:20:16.377883 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:16.377856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:21.373084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:21.373005    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:38.857320 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:38.857286    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:38.857870 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:38.857544    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" podUID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" containerName="maas-api" containerID="cri-o://a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e" gracePeriod=30
Jul 29 21:20:38.904109 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:38.904077    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:38.904402 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:38.904335    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" podUID="201b1497-f53a-47ed-94a4-3ce419c298f0" containerName="payload-processing" containerID="cri-o://2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598" gracePeriod=30
Jul 29 21:20:39.202626 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.202601    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:39.205646 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.205627    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:39.230074 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230044    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca\") pod \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") "
Jul 29 21:20:39.230224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhrxh\" (UniqueName: \"kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh\") pod \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") "
Jul 29 21:20:39.230224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-866vc\" (UniqueName: \"kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc\") pod \"201b1497-f53a-47ed-94a4-3ce419c298f0\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") "
Jul 29 21:20:39.230343 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume\") pod \"201b1497-f53a-47ed-94a4-3ce419c298f0\" (UID: \"201b1497-f53a-47ed-94a4-3ce419c298f0\") "
Jul 29 21:20:39.230343 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230270    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") pod \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\" (UID: \"023ced4c-1b52-4673-8d7d-d4db6ac0eac2\") "
Jul 29 21:20:39.230494 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "023ced4c-1b52-4673-8d7d-d4db6ac0eac2" (UID: "023ced4c-1b52-4673-8d7d-d4db6ac0eac2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:20:39.230691 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230669    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:39.230957 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.230693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "201b1497-f53a-47ed-94a4-3ce419c298f0" (UID: "201b1497-f53a-47ed-94a4-3ce419c298f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:20:39.232490 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.232456    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc" (OuterVolumeSpecName: "kube-api-access-866vc") pod "201b1497-f53a-47ed-94a4-3ce419c298f0" (UID: "201b1497-f53a-47ed-94a4-3ce419c298f0"). InnerVolumeSpecName "kube-api-access-866vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:20:39.232588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.232503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "023ced4c-1b52-4673-8d7d-d4db6ac0eac2" (UID: "023ced4c-1b52-4673-8d7d-d4db6ac0eac2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:20:39.232588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.232532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh" (OuterVolumeSpecName: "kube-api-access-hhrxh") pod "023ced4c-1b52-4673-8d7d-d4db6ac0eac2" (UID: "023ced4c-1b52-4673-8d7d-d4db6ac0eac2"). InnerVolumeSpecName "kube-api-access-hhrxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:20:39.332409 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.332361    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/201b1497-f53a-47ed-94a4-3ce419c298f0-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:39.332584 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.332427    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:39.332584 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.332442    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhrxh\" (UniqueName: \"kubernetes.io/projected/023ced4c-1b52-4673-8d7d-d4db6ac0eac2-kube-api-access-hhrxh\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:39.332584 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.332457    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-866vc\" (UniqueName: \"kubernetes.io/projected/201b1497-f53a-47ed-94a4-3ce419c298f0-kube-api-access-866vc\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:20:39.515137 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.515100    2581 generic.go:356] "Generic (PLEG): container finished" podID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" containerID="a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e" exitCode=0
Jul 29 21:20:39.515332 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.515146    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"
Jul 29 21:20:39.515332 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.515177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" event={"ID":"023ced4c-1b52-4673-8d7d-d4db6ac0eac2","Type":"ContainerDied","Data":"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"}
Jul 29 21:20:39.515332 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.515229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p" event={"ID":"023ced4c-1b52-4673-8d7d-d4db6ac0eac2","Type":"ContainerDied","Data":"45c4afa0082d4aec3ea8face0ab4a90096ff266ecf5b304d0682adef8af1b507"}
Jul 29 21:20:39.515332 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.515261    2581 scope.go:117] "RemoveContainer" containerID="a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"
Jul 29 21:20:39.516544 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.516527    2581 generic.go:356] "Generic (PLEG): container finished" podID="201b1497-f53a-47ed-94a4-3ce419c298f0" containerID="2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598" exitCode=0
Jul 29 21:20:39.516626 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.516574    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"
Jul 29 21:20:39.516626 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.516596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" event={"ID":"201b1497-f53a-47ed-94a4-3ce419c298f0","Type":"ContainerDied","Data":"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"}
Jul 29 21:20:39.516701 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.516642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj" event={"ID":"201b1497-f53a-47ed-94a4-3ce419c298f0","Type":"ContainerDied","Data":"c170c46bef019a64101b98a3e7c748abe8776d685aa8864e868a8c9cfcfd4b34"}
Jul 29 21:20:39.525780 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.525757    2581 scope.go:117] "RemoveContainer" containerID="a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"
Jul 29 21:20:39.526125 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:20:39.526083    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e\": container with ID starting with a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e not found: ID does not exist" containerID="a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"
Jul 29 21:20:39.526277 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.526133    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e"} err="failed to get container status \"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e\": rpc error: code = NotFound desc = could not find container \"a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e\": container with ID starting with a0704bd0db5032963ee25e14d7812733f246448dc8ce1c9fc67b7a6a8663f48e not found: ID does not exist"
Jul 29 21:20:39.526277 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.526150    2581 scope.go:117] "RemoveContainer" containerID="2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"
Jul 29 21:20:39.534734 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.534719    2581 scope.go:117] "RemoveContainer" containerID="2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"
Jul 29 21:20:39.535039 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:20:39.535021    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598\": container with ID starting with 2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598 not found: ID does not exist" containerID="2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"
Jul 29 21:20:39.535090 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.535046    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598"} err="failed to get container status \"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598\": rpc error: code = NotFound desc = could not find container \"2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598\": container with ID starting with 2eab95fdd44b0d17053358586d52d089dc525ff28d7018feb143cb6a32fd7598 not found: ID does not exist"
Jul 29 21:20:39.540620 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.540600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:39.545939 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.545918    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f1022d1c-c8445cf85-vpt7p"]
Jul 29 21:20:39.557155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.557132    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:39.559452 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:39.559432    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-f1022d1c-7cb67c7f54-cf4gj"]
Jul 29 21:20:40.067414 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:40.067352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:40.168075 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:40.167995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:40.263589 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:40.263562    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" path="/var/lib/kubelet/pods/023ced4c-1b52-4673-8d7d-d4db6ac0eac2/volumes"
Jul 29 21:20:40.263941 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:40.263929    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="201b1497-f53a-47ed-94a4-3ce419c298f0" path="/var/lib/kubelet/pods/201b1497-f53a-47ed-94a4-3ce419c298f0/volumes"
Jul 29 21:20:41.659555 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:41.659514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:47.519427 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:47.519375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:47.638302 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:47.638266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:48.681976 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:48.681931    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:48.749528 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:20:48.749496    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:14.251446 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:14.251410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:20.788560 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:20.788525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:20.894682 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:20.894643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:20.989363 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:20.989325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:36.982631 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:36.982593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:42.657986 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:42.657949    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:42.716717 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:42.716685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:42.809226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:42.809159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:42.902729 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:42.902699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:53.510522 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:53.510438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:54.291040 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:21:54.291000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:00.536082 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:00.536043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:00.646797 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:00.646765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:00.763360 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:00.763311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:00.881596 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:00.881516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:08.341359 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:08.341326    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:08.477836 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:08.477803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:10.471375 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:10.471305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:16.437941 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:16.437905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:16.537712 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:16.537674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:17.248820 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:17.248776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:23.202219 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:23.202182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:23.302409 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:23.302346    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:23.488897 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:23.488809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:24.057144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:24.056857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:24.654342 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:24.654303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:27.615834 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:27.615797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:27.713192 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:27.713150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:28.738160 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738123    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:22:28.738697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738665    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" containerName="maas-api"
Jul 29 21:22:28.738697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738682    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" containerName="maas-api"
Jul 29 21:22:28.738824 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738703    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201b1497-f53a-47ed-94a4-3ce419c298f0" containerName="payload-processing"
Jul 29 21:22:28.738824 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738709    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="201b1497-f53a-47ed-94a4-3ce419c298f0" containerName="payload-processing"
Jul 29 21:22:28.738824 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738771    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="201b1497-f53a-47ed-94a4-3ce419c298f0" containerName="payload-processing"
Jul 29 21:22:28.738824 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.738781    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="023ced4c-1b52-4673-8d7d-d4db6ac0eac2" containerName="maas-api"
Jul 29 21:22:28.742306 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.742288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.744490 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.744467    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ccbf66ba-serving-cert\""
Jul 29 21:22:28.755974 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.755947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:22:28.783194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.783156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:28.865996 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.865958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:28.885217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.885192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.885349 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.885227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.885419 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.885369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn4sh\" (UniqueName: \"kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.986349 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.986310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn4sh\" (UniqueName: \"kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.986555 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.986416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.986555 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.986444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.987059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.987036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:28.989092 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:28.989038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:29.002366 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:29.002336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn4sh\" (UniqueName: \"kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh\") pod \"maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:29.053164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:29.053129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:29.203196 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:29.203165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:22:29.205789 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:22:29.205762    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc39764f_6e45_4287_9804_75706ea31533.slice/crio-8bcaf0b891e9f5e8f860ab02d10e2f3a9795ec463afb12ddaaabce8240381328 WatchSource:0}: Error finding container 8bcaf0b891e9f5e8f860ab02d10e2f3a9795ec463afb12ddaaabce8240381328: Status 404 returned error can't find the container with id 8bcaf0b891e9f5e8f860ab02d10e2f3a9795ec463afb12ddaaabce8240381328
Jul 29 21:22:29.280372 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:29.280335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:29.383898 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:29.383859    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:30.095253 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.095220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" event={"ID":"bc39764f-6e45-4287-9804-75706ea31533","Type":"ContainerStarted","Data":"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"}
Jul 29 21:22:30.095669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.095267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" event={"ID":"bc39764f-6e45-4287-9804-75706ea31533","Type":"ContainerStarted","Data":"8bcaf0b891e9f5e8f860ab02d10e2f3a9795ec463afb12ddaaabce8240381328"}
Jul 29 21:22:30.095669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.095329    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:30.116050 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.115955    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" podStartSLOduration=1.4624749750000001 podStartE2EDuration="2.115940663s" totalImagesPullingTime="653.465688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:22:30.115312059 +0000 UTC m=+2100.364549575" watchObservedRunningTime="2026-07-29 21:22:30.115940663 +0000 UTC m=+2100.365178193"
Jul 29 21:22:30.746096 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.746067    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:22:30.757925 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:30.757363    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:22:36.105657 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:36.105631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:22:52.613202 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:52.613158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:53.392971 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:53.392938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:53.481806 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:53.481768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:53.799329 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:53.799294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:59.576436 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:59.576376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:59.647004 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:22:59.646965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.521125 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.521084    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:23:10.521601 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.521348    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" podUID="bc39764f-6e45-4287-9804-75706ea31533" containerName="maas-api" containerID="cri-o://bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007" gracePeriod=30
Jul 29 21:23:10.543190 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.543128    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.644973 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.644942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.722179 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.722144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.776154 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.776076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:23:10.887697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.887667    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca\") pod \"bc39764f-6e45-4287-9804-75706ea31533\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") "
Jul 29 21:23:10.887697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.887701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn4sh\" (UniqueName: \"kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh\") pod \"bc39764f-6e45-4287-9804-75706ea31533\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") "
Jul 29 21:23:10.887902 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.887731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls\") pod \"bc39764f-6e45-4287-9804-75706ea31533\" (UID: \"bc39764f-6e45-4287-9804-75706ea31533\") "
Jul 29 21:23:10.888143 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.888120    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc39764f-6e45-4287-9804-75706ea31533" (UID: "bc39764f-6e45-4287-9804-75706ea31533"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:23:10.890036 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.890006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh" (OuterVolumeSpecName: "kube-api-access-xn4sh") pod "bc39764f-6e45-4287-9804-75706ea31533" (UID: "bc39764f-6e45-4287-9804-75706ea31533"). InnerVolumeSpecName "kube-api-access-xn4sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:23:10.890148 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.890068    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc39764f-6e45-4287-9804-75706ea31533" (UID: "bc39764f-6e45-4287-9804-75706ea31533"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:23:10.989510 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.989479    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc39764f-6e45-4287-9804-75706ea31533-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:23:10.989510 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.989505    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn4sh\" (UniqueName: \"kubernetes.io/projected/bc39764f-6e45-4287-9804-75706ea31533-kube-api-access-xn4sh\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:23:10.989510 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:10.989516    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc39764f-6e45-4287-9804-75706ea31533-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:23:11.314514 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.314480    2581 generic.go:356] "Generic (PLEG): container finished" podID="bc39764f-6e45-4287-9804-75706ea31533" containerID="bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007" exitCode=0
Jul 29 21:23:11.314693 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.314521    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"
Jul 29 21:23:11.314693 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.314561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" event={"ID":"bc39764f-6e45-4287-9804-75706ea31533","Type":"ContainerDied","Data":"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"}
Jul 29 21:23:11.314693 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.314616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6" event={"ID":"bc39764f-6e45-4287-9804-75706ea31533","Type":"ContainerDied","Data":"8bcaf0b891e9f5e8f860ab02d10e2f3a9795ec463afb12ddaaabce8240381328"}
Jul 29 21:23:11.314693 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.314637    2581 scope.go:117] "RemoveContainer" containerID="bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"
Jul 29 21:23:11.316775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.316747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:11.324411 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.324377    2581 scope.go:117] "RemoveContainer" containerID="bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"
Jul 29 21:23:11.324656 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:23:11.324637    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007\": container with ID starting with bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007 not found: ID does not exist" containerID="bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"
Jul 29 21:23:11.324708 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.324664    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007"} err="failed to get container status \"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007\": rpc error: code = NotFound desc = could not find container \"bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007\": container with ID starting with bece9c1b7835a051ab9ba0f72b16a379a2508af50ccb0e82b19b098dec0da007 not found: ID does not exist"
Jul 29 21:23:11.337841 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.337812    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:23:11.345733 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.345713    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ccbf66ba-548545bd7c-l82b6"]
Jul 29 21:23:11.390894 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:11.390869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:12.263689 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:12.263657    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc39764f-6e45-4287-9804-75706ea31533" path="/var/lib/kubelet/pods/bc39764f-6e45-4287-9804-75706ea31533/volumes"
Jul 29 21:23:18.329081 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:18.329043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:18.415299 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:18.415262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:18.707023 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:18.706923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:18.894101 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:18.894057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:19.025873 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:19.025841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:19.119974 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:19.119940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:19.269007 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:19.268972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:24.901110 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:24.901025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:24.969977 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:24.969937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:25.091555 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:25.091525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:25.193504 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:25.193420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:25.288193 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:25.288161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:30.996254 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:30.996216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:31.098181 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:31.098148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:37.851733 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:37.851692    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:38.182526 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:38.182424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:38.335831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:38.335794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:40.884953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:40.884913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:40.980957 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:40.980888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:41.979062 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.979028    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:23:41.979536 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.979500    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc39764f-6e45-4287-9804-75706ea31533" containerName="maas-api"
Jul 29 21:23:41.979536 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.979513    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc39764f-6e45-4287-9804-75706ea31533" containerName="maas-api"
Jul 29 21:23:41.979643 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.979573    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc39764f-6e45-4287-9804-75706ea31533" containerName="maas-api"
Jul 29 21:23:41.982684 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.982666    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:41.985034 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.985014    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ac04a8f1-serving-cert\""
Jul 29 21:23:41.994937 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:41.994914    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:23:42.017645 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.017620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:42.077769 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.077738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.077962 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.077827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.077962 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.077919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lv6z\" (UniqueName: \"kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.179115 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.179077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lv6z\" (UniqueName: \"kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.179306 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.179150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.179306 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.179233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.179478 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:23:42.179422    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-serving-cert: secret "maas-api-e2e-mt-ac04a8f1-serving-cert" not found
Jul 29 21:23:42.179536 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:23:42.179523    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls podName:8efa7d8b-159f-430b-80f3-5ab0d7ae813f nodeName:}" failed. No retries permitted until 2026-07-29 21:23:42.679499121 +0000 UTC m=+2172.928736620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls") pod "maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" (UID: "8efa7d8b-159f-430b-80f3-5ab0d7ae813f") : secret "maas-api-e2e-mt-ac04a8f1-serving-cert" not found
Jul 29 21:23:42.179809 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.179788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.189011 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.188979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lv6z\" (UniqueName: \"kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.428298 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.428263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:42.684077 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.683998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.686615 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.686588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") pod \"maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:42.732473 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.732439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:42.828372 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.828338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:42.895304 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:42.895275    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:43.040697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:43.040675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:23:43.042925 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:23:43.042900    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efa7d8b_159f_430b_80f3_5ab0d7ae813f.slice/crio-4ffb0195b0742e19fb086e3d0b197fb90a0710f75eb89fc7c778a83f0a2bfa31 WatchSource:0}: Error finding container 4ffb0195b0742e19fb086e3d0b197fb90a0710f75eb89fc7c778a83f0a2bfa31: Status 404 returned error can't find the container with id 4ffb0195b0742e19fb086e3d0b197fb90a0710f75eb89fc7c778a83f0a2bfa31
Jul 29 21:23:43.044634 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:43.044618    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:23:43.482801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:43.482766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" event={"ID":"8efa7d8b-159f-430b-80f3-5ab0d7ae813f","Type":"ContainerStarted","Data":"4ffb0195b0742e19fb086e3d0b197fb90a0710f75eb89fc7c778a83f0a2bfa31"}
Jul 29 21:23:44.489771 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:44.489735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" event={"ID":"8efa7d8b-159f-430b-80f3-5ab0d7ae813f","Type":"ContainerStarted","Data":"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"}
Jul 29 21:23:44.490285 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:44.489869    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:44.508549 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:44.508501    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" podStartSLOduration=3.102337134 podStartE2EDuration="3.508485544s" totalImagesPullingTime="406.14841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:23:44.507223649 +0000 UTC m=+2174.756461165" watchObservedRunningTime="2026-07-29 21:23:44.508485544 +0000 UTC m=+2174.757723059"
Jul 29 21:23:50.500832 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:50.500794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:23:59.049324 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:23:59.049292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:05.875898 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:05.875865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:11.376509 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:11.376465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:11.907263 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:11.907227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:12.007539 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:12.007491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:50.771112 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:50.771018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:24:50.771714 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:50.771288    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" podUID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" containerName="maas-api" containerID="cri-o://65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc" gracePeriod=30
Jul 29 21:24:50.798639 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:50.798605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:50.886742 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:50.886700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:50.989219 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:50.989185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:51.025441 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.025366    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:24:51.146067 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.146036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:51.166108 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.166080    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca\") pod \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") "
Jul 29 21:24:51.166266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.166221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") pod \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") "
Jul 29 21:24:51.166331 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.166275    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lv6z\" (UniqueName: \"kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z\") pod \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\" (UID: \"8efa7d8b-159f-430b-80f3-5ab0d7ae813f\") "
Jul 29 21:24:51.166556 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.166530    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8efa7d8b-159f-430b-80f3-5ab0d7ae813f" (UID: "8efa7d8b-159f-430b-80f3-5ab0d7ae813f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:24:51.166695 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.166672    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:24:51.168760 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.168737    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z" (OuterVolumeSpecName: "kube-api-access-2lv6z") pod "8efa7d8b-159f-430b-80f3-5ab0d7ae813f" (UID: "8efa7d8b-159f-430b-80f3-5ab0d7ae813f"). InnerVolumeSpecName "kube-api-access-2lv6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:51.168989 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.168965    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8efa7d8b-159f-430b-80f3-5ab0d7ae813f" (UID: "8efa7d8b-159f-430b-80f3-5ab0d7ae813f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:24:51.268059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.268030    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:24:51.268059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.268054    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lv6z\" (UniqueName: \"kubernetes.io/projected/8efa7d8b-159f-430b-80f3-5ab0d7ae813f-kube-api-access-2lv6z\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:24:51.831103 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.831066    2581 generic.go:356] "Generic (PLEG): container finished" podID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" containerID="65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc" exitCode=0
Jul 29 21:24:51.831562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.831122    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"
Jul 29 21:24:51.831562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.831145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" event={"ID":"8efa7d8b-159f-430b-80f3-5ab0d7ae813f","Type":"ContainerDied","Data":"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"}
Jul 29 21:24:51.831562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.831202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz" event={"ID":"8efa7d8b-159f-430b-80f3-5ab0d7ae813f","Type":"ContainerDied","Data":"4ffb0195b0742e19fb086e3d0b197fb90a0710f75eb89fc7c778a83f0a2bfa31"}
Jul 29 21:24:51.831562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.831226    2581 scope.go:117] "RemoveContainer" containerID="65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"
Jul 29 21:24:51.840645 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.840612    2581 scope.go:117] "RemoveContainer" containerID="65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"
Jul 29 21:24:51.840935 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:24:51.840916    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc\": container with ID starting with 65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc not found: ID does not exist" containerID="65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"
Jul 29 21:24:51.841009 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.840943    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc"} err="failed to get container status \"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc\": rpc error: code = NotFound desc = could not find container \"65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc\": container with ID starting with 65f06428dda0eb82b2affcad8716776ea5068d986a3cd4d8d140fa266ea47edc not found: ID does not exist"
Jul 29 21:24:51.856796 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.856769    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:24:51.860586 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:51.860560    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ac04a8f1-69d48f687d-6z7kz"]
Jul 29 21:24:52.264267 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:52.264233    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" path="/var/lib/kubelet/pods/8efa7d8b-159f-430b-80f3-5ab0d7ae813f/volumes"
Jul 29 21:24:54.405034 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:54.405002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:56.074412 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:56.074349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:59.474743 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:24:59.474695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:28.658034 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:28.657995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:35.811573 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:35.811535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:37.455328 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:37.455283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:43.307798 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:43.307765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:43.424036 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:43.424002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:43.599509 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:43.599422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:43.813316 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:43.813286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:44.904058 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:44.904016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:44.997947 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:44.997908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:45.102829 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:45.102792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:45.897000 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:45.896967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:45.978776 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:45.978729    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:47.578282 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:47.578247    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:53.822023 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:53.821990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:53.890794 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:53.890758    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:54.594435 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:25:54.594348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:01.386120 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:01.386070    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:01.569991 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:01.569951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:01.779193 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:01.779161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:12.684414 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:12.684350    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:12.785559 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:12.785519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:33.447425 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:33.447312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:40.689111 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:40.689076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:40.762410 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:40.762348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:41.043773 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:41.043728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:41.417412 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:41.417288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:41.565539 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:41.565507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:43.963140 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:43.963084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:44.053516 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:44.053487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:45.170460 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.170426    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:26:45.170857 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.170834    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" containerName="maas-api"
Jul 29 21:26:45.170857 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.170845    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" containerName="maas-api"
Jul 29 21:26:45.170979 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.170913    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efa7d8b-159f-430b-80f3-5ab0d7ae813f" containerName="maas-api"
Jul 29 21:26:45.174053 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.174034    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.176489 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.176459    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4d12bd-serving-cert\""
Jul 29 21:26:45.184787 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.184764    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:26:45.253451 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.253422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:45.270843 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.270802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ndmh\" (UniqueName: \"kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.271002 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.270851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.271064 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.271006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.364203 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.364135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:45.371983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.371959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ndmh\" (UniqueName: \"kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.372076 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.372000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.372136 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.372119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.372856 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.372838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.374794 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.374774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.380160 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.380137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ndmh\" (UniqueName: \"kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh\") pod \"maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.486355 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.486270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:45.629982 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.629943    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:26:45.632508 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:26:45.632464    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5dd6f66_d61a_425b_b991_b68a5cfdf2aa.slice/crio-ff8ea47137c60116f82ca9bdbc4e7b43aaf10c50eadf4a292b08f6873460a9dd WatchSource:0}: Error finding container ff8ea47137c60116f82ca9bdbc4e7b43aaf10c50eadf4a292b08f6873460a9dd: Status 404 returned error can't find the container with id ff8ea47137c60116f82ca9bdbc4e7b43aaf10c50eadf4a292b08f6873460a9dd
Jul 29 21:26:45.744588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.744505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:45.848752 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:45.848715    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:46.411898 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:46.411803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" event={"ID":"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa","Type":"ContainerStarted","Data":"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"}
Jul 29 21:26:46.411898 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:46.411854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" event={"ID":"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa","Type":"ContainerStarted","Data":"ff8ea47137c60116f82ca9bdbc4e7b43aaf10c50eadf4a292b08f6873460a9dd"}
Jul 29 21:26:46.411898 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:46.411896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:26:46.432265 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:46.432207    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" podStartSLOduration=0.938333096 podStartE2EDuration="1.432190787s" totalImagesPullingTime="493.857691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:46.430769948 +0000 UTC m=+2356.680007465" watchObservedRunningTime="2026-07-29 21:26:46.432190787 +0000 UTC m=+2356.681428301"
Jul 29 21:26:52.422772 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:26:52.422741    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:27:09.006835 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:09.006802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:09.371737 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:09.371619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:09.906206 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:09.906174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:10.006836 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:10.006786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:13.207108 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:13.207074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:13.306705 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:13.306663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:14.364854 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.364811    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:27:14.368820 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.368796    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.371271 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.371249    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-60a3ea-serving-cert\""
Jul 29 21:27:14.381984 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.381957    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:27:14.474552 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.474513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.474782 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.474585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7bb9\" (UniqueName: \"kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.474782 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.474681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.510681 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.510646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:14.575339 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.575300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.575531 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.575372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7bb9\" (UniqueName: \"kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.575531 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.575452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.575531 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:27:14.575494    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-serving-cert: secret "maas-api-e2e-inf-b-60a3ea-serving-cert" not found
Jul 29 21:27:14.575649 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:27:14.575581    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls podName:7bc5aa1b-bcbe-4647-8749-a985c1e77c2e nodeName:}" failed. No retries permitted until 2026-07-29 21:27:15.075559449 +0000 UTC m=+2385.324796943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls") pod "maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" (UID: "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e") : secret "maas-api-e2e-inf-b-60a3ea-serving-cert" not found
Jul 29 21:27:14.576121 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.576103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.585718 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.585689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7bb9\" (UniqueName: \"kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:14.866633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.866605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:14.975803 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.975767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:14.976747 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.976723    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:27:14.980499 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.980480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:14.982730 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.982710    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-60a3ea\""
Jul 29 21:27:14.982845 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.982764    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-60a3ea-dockercfg-99hz4\""
Jul 29 21:27:14.994292 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:14.994269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:27:15.080909 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.080874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6zvt\" (UniqueName: \"kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.081116 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.080933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:15.081116 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.080997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.081242 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:27:15.081225    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-serving-cert: secret "maas-api-e2e-inf-b-60a3ea-serving-cert" not found
Jul 29 21:27:15.081313 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:27:15.081300    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls podName:7bc5aa1b-bcbe-4647-8749-a985c1e77c2e nodeName:}" failed. No retries permitted until 2026-07-29 21:27:16.081281024 +0000 UTC m=+2386.330518518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls") pod "maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" (UID: "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e") : secret "maas-api-e2e-inf-b-60a3ea-serving-cert" not found
Jul 29 21:27:15.182448 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.182338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zvt\" (UniqueName: \"kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.182625 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.182504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.183173 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.183148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.190235 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.190210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6zvt\" (UniqueName: \"kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt\") pod \"payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.291692 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.291653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:15.333717 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.333681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:15.643735 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:15.643710    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:27:15.645056 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:27:15.645033    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3596e154_1dbb_4ce7_b2ae_18a8a347bd2f.slice/crio-538a6ce788bacd2f9b6649ea792a057c983aec4e022455ab152e5e20ad9c6042 WatchSource:0}: Error finding container 538a6ce788bacd2f9b6649ea792a057c983aec4e022455ab152e5e20ad9c6042: Status 404 returned error can't find the container with id 538a6ce788bacd2f9b6649ea792a057c983aec4e022455ab152e5e20ad9c6042
Jul 29 21:27:16.092142 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.092092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:16.095127 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.095098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:16.182794 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.182759    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:16.324473 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.324448    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:27:16.326839 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:27:16.326817    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc5aa1b_bcbe_4647_8749_a985c1e77c2e.slice/crio-bd8874aa97891d1132ce6b9ba17362452caaec459b095cd63e71834c31d34d41 WatchSource:0}: Error finding container bd8874aa97891d1132ce6b9ba17362452caaec459b095cd63e71834c31d34d41: Status 404 returned error can't find the container with id bd8874aa97891d1132ce6b9ba17362452caaec459b095cd63e71834c31d34d41
Jul 29 21:27:16.561251 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.561219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" event={"ID":"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e","Type":"ContainerStarted","Data":"bd8874aa97891d1132ce6b9ba17362452caaec459b095cd63e71834c31d34d41"}
Jul 29 21:27:16.562775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.562742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" event={"ID":"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f","Type":"ContainerStarted","Data":"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"}
Jul 29 21:27:16.562933 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.562790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" event={"ID":"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f","Type":"ContainerStarted","Data":"538a6ce788bacd2f9b6649ea792a057c983aec4e022455ab152e5e20ad9c6042"}
Jul 29 21:27:16.562933 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.562853    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:16.580377 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:16.580320    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" podStartSLOduration=2.580306384 podStartE2EDuration="2.580306384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:27:16.577818323 +0000 UTC m=+2386.827055872" watchObservedRunningTime="2026-07-29 21:27:16.580306384 +0000 UTC m=+2386.829543899"
Jul 29 21:27:17.569547 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:17.569509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" event={"ID":"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e","Type":"ContainerStarted","Data":"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"}
Jul 29 21:27:17.570005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:17.569723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:17.591486 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:17.591414    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" podStartSLOduration=3.104119542 podStartE2EDuration="3.59136167s" totalImagesPullingTime="487.242128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:27:17.590844789 +0000 UTC m=+2387.840082341" watchObservedRunningTime="2026-07-29 21:27:17.59136167 +0000 UTC m=+2387.840599168"
Jul 29 21:27:23.579808 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:23.579779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:27:27.572178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:27.572146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:27:30.784168 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:30.784064    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:27:30.804498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:30.792772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:27:38.254622 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:38.254582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:47.925144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:47.925114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:48.009097 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:48.009054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:50.413835 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:27:50.413756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:08.996968 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:28:08.996862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:58.044699 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:28:58.044657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:58.991734 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:28:58.991704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:09.572325 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.572288    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:29:09.572881 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.572522    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" podUID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" containerName="maas-api" containerID="cri-o://38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1" gracePeriod=30
Jul 29 21:29:09.586951 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.586911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:09.841700 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.841677    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:29:09.916489 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.916457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls\") pod \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") "
Jul 29 21:29:09.916489 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.916496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ndmh\" (UniqueName: \"kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh\") pod \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") "
Jul 29 21:29:09.916687 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.916600    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca\") pod \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\" (UID: \"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa\") "
Jul 29 21:29:09.916983 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.916955    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" (UID: "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:29:09.918631 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.918606    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" (UID: "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:29:09.918823 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:09.918806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh" (OuterVolumeSpecName: "kube-api-access-5ndmh") pod "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" (UID: "a5dd6f66-d61a-425b-b991-b68a5cfdf2aa"). InnerVolumeSpecName "kube-api-access-5ndmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:29:10.017605 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.017568    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:10.017605 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.017600    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:10.017605 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.017609    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ndmh\" (UniqueName: \"kubernetes.io/projected/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa-kube-api-access-5ndmh\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:10.157771 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.157684    2581 generic.go:356] "Generic (PLEG): container finished" podID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" containerID="38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1" exitCode=0
Jul 29 21:29:10.157771 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.157731    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"
Jul 29 21:29:10.157951 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.157777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" event={"ID":"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa","Type":"ContainerDied","Data":"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"}
Jul 29 21:29:10.157951 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.157837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4" event={"ID":"a5dd6f66-d61a-425b-b991-b68a5cfdf2aa","Type":"ContainerDied","Data":"ff8ea47137c60116f82ca9bdbc4e7b43aaf10c50eadf4a292b08f6873460a9dd"}
Jul 29 21:29:10.157951 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.157868    2581 scope.go:117] "RemoveContainer" containerID="38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"
Jul 29 21:29:10.167177 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.167156    2581 scope.go:117] "RemoveContainer" containerID="38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"
Jul 29 21:29:10.167450 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:29:10.167429    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1\": container with ID starting with 38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1 not found: ID does not exist" containerID="38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"
Jul 29 21:29:10.167515 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.167460    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1"} err="failed to get container status \"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1\": rpc error: code = NotFound desc = could not find container \"38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1\": container with ID starting with 38dc69931efbadb8f0353bbfdbf5f25596a3abbf1780fa04b7748ae4964271a1 not found: ID does not exist"
Jul 29 21:29:10.180990 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.180962    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:29:10.187079 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.187053    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d12bd-55469d8767-vbst4"]
Jul 29 21:29:10.270178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:10.270145    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" path="/var/lib/kubelet/pods/a5dd6f66-d61a-425b-b991-b68a5cfdf2aa/volumes"
Jul 29 21:29:11.683619 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:11.683584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:15.994781 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:15.994744    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:16.061419 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:16.061368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:16.278976 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:16.278919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:16.504642 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:16.504604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.044085 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.044000    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:29:27.044541 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.044254    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" podUID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" containerName="maas-api" containerID="cri-o://c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518" gracePeriod=30
Jul 29 21:29:27.131500 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.131459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.198856 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.198824    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:29:27.199068 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.199043    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" podUID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" containerName="payload-pre-processing" containerID="cri-o://e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4" gracePeriod=30
Jul 29 21:29:27.281813 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.281778    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.378122 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.378045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.473753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.473727    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.480319 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.480291    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:29:27.521503 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.521478    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:29:27.597437 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.597324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6zvt\" (UniqueName: \"kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt\") pod \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") "
Jul 29 21:29:27.597437 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.597434    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume\") pod \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\" (UID: \"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f\") "
Jul 29 21:29:27.597832 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.597810    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" (UID: "3596e154-1dbb-4ce7-b2ae-18a8a347bd2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:29:27.599656 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.599636    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt" (OuterVolumeSpecName: "kube-api-access-q6zvt") pod "3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" (UID: "3596e154-1dbb-4ce7-b2ae-18a8a347bd2f"). InnerVolumeSpecName "kube-api-access-q6zvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:29:27.698012 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.697971    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7bb9\" (UniqueName: \"kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9\") pod \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") "
Jul 29 21:29:27.698170 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.698107    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca\") pod \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") "
Jul 29 21:29:27.698170 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.698153    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") pod \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\" (UID: \"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e\") "
Jul 29 21:29:27.698405 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.698368    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6zvt\" (UniqueName: \"kubernetes.io/projected/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-kube-api-access-q6zvt\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:27.698508 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.698443    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:27.698571 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.698501    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" (UID: "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:29:27.700373 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.700342    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" (UID: "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:29:27.700373 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.700353    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9" (OuterVolumeSpecName: "kube-api-access-j7bb9") pod "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" (UID: "7bc5aa1b-bcbe-4647-8749-a985c1e77c2e"). InnerVolumeSpecName "kube-api-access-j7bb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:29:27.776247 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.776207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:27.799563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.799529    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7bb9\" (UniqueName: \"kubernetes.io/projected/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-kube-api-access-j7bb9\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:27.799563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.799561    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:27.799563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:27.799571    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:29:28.253671 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.253636    2581 generic.go:356] "Generic (PLEG): container finished" podID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" containerID="c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518" exitCode=0
Jul 29 21:29:28.254107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.253690    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"
Jul 29 21:29:28.254107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.253713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" event={"ID":"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e","Type":"ContainerDied","Data":"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"}
Jul 29 21:29:28.254107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.253792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb" event={"ID":"7bc5aa1b-bcbe-4647-8749-a985c1e77c2e","Type":"ContainerDied","Data":"bd8874aa97891d1132ce6b9ba17362452caaec459b095cd63e71834c31d34d41"}
Jul 29 21:29:28.254107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.253825    2581 scope.go:117] "RemoveContainer" containerID="c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"
Jul 29 21:29:28.255260 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.254996    2581 generic.go:356] "Generic (PLEG): container finished" podID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" containerID="e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4" exitCode=0
Jul 29 21:29:28.255260 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.255056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"
Jul 29 21:29:28.255260 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.255085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" event={"ID":"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f","Type":"ContainerDied","Data":"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"}
Jul 29 21:29:28.255260 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.255133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n" event={"ID":"3596e154-1dbb-4ce7-b2ae-18a8a347bd2f","Type":"ContainerDied","Data":"538a6ce788bacd2f9b6649ea792a057c983aec4e022455ab152e5e20ad9c6042"}
Jul 29 21:29:28.267527 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.266696    2581 scope.go:117] "RemoveContainer" containerID="c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"
Jul 29 21:29:28.268174 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:29:28.268144    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518\": container with ID starting with c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518 not found: ID does not exist" containerID="c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"
Jul 29 21:29:28.268307 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.268179    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518"} err="failed to get container status \"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518\": rpc error: code = NotFound desc = could not find container \"c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518\": container with ID starting with c64d05e154d99fb5a2c8c079fbbccdf057ea7991df5206a5e7da546b71b2f518 not found: ID does not exist"
Jul 29 21:29:28.268307 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.268203    2581 scope.go:117] "RemoveContainer" containerID="e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"
Jul 29 21:29:28.276957 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.276941    2581 scope.go:117] "RemoveContainer" containerID="e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"
Jul 29 21:29:28.277238 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:29:28.277219    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4\": container with ID starting with e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4 not found: ID does not exist" containerID="e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"
Jul 29 21:29:28.277308 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.277246    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4"} err="failed to get container status \"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4\": rpc error: code = NotFound desc = could not find container \"e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4\": container with ID starting with e22aacb29fae9eaadfdba191ff91aa6cfc51643c7f6f6c1304bd21e47e8238b4 not found: ID does not exist"
Jul 29 21:29:28.292088 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.292053    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:29:28.296556 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.296525    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-60a3ea-6d59cdcb85-m8rvb"]
Jul 29 21:29:28.310268 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.310236    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:29:28.319305 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:28.319275    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-60a3ea-f8f974c6d-59f7n"]
Jul 29 21:29:30.263843 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:30.263808    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" path="/var/lib/kubelet/pods/3596e154-1dbb-4ce7-b2ae-18a8a347bd2f/volumes"
Jul 29 21:29:30.264358 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:30.264342    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" path="/var/lib/kubelet/pods/7bc5aa1b-bcbe-4647-8749-a985c1e77c2e/volumes"
Jul 29 21:29:34.654645 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:34.654595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:34.755295 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:34.755259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:35.040650 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:35.040621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:35.315252 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:35.315170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:37.979920 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:37.979887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:38.077986 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:38.077953    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:39.074832 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.074800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:29:39.075428 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075410    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" containerName="maas-api"
Jul 29 21:29:39.075498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075430    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" containerName="maas-api"
Jul 29 21:29:39.075498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075458    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" containerName="maas-api"
Jul 29 21:29:39.075498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075467    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" containerName="maas-api"
Jul 29 21:29:39.075498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075491    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" containerName="payload-pre-processing"
Jul 29 21:29:39.075498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075499    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" containerName="payload-pre-processing"
Jul 29 21:29:39.075765 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075591    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5dd6f66-d61a-425b-b991-b68a5cfdf2aa" containerName="maas-api"
Jul 29 21:29:39.075765 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075605    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3596e154-1dbb-4ce7-b2ae-18a8a347bd2f" containerName="payload-pre-processing"
Jul 29 21:29:39.075765 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.075613    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc5aa1b-bcbe-4647-8749-a985c1e77c2e" containerName="maas-api"
Jul 29 21:29:39.079165 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.079144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.081563 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.081541    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a7e568-serving-cert\""
Jul 29 21:29:39.088131 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.088070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:29:39.108822 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.108789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.109037 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.108867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.109037 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.108923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhk7\" (UniqueName: \"kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.112085 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.112060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:39.210166 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.210128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhk7\" (UniqueName: \"kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.210358 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.210225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.210358 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.210309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.210553 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:29:39.210474    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-serving-cert: secret "maas-api-e2e-api-a-a7e568-serving-cert" not found
Jul 29 21:29:39.210553 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:29:39.210542    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls podName:e36422d9-b6b6-41c5-935a-52c5c1f53fa9 nodeName:}" failed. No retries permitted until 2026-07-29 21:29:39.710522009 +0000 UTC m=+2529.959759506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls") pod "maas-api-e2e-api-a-a7e568-694646f547-wh2fm" (UID: "e36422d9-b6b6-41c5-935a-52c5c1f53fa9") : secret "maas-api-e2e-api-a-a7e568-serving-cert" not found
Jul 29 21:29:39.211009 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.210992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.219314 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.219279    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhk7\" (UniqueName: \"kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.653492 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.653462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:39.718060 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.718026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.720628 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.720595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e568-694646f547-wh2fm\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:39.751223 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.751188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:39.992550 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:39.992461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:40.134030 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:40.133998    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:29:40.136942 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:29:40.136910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36422d9_b6b6_41c5_935a_52c5c1f53fa9.slice/crio-18a577ed031bf284bd15e88adf076b40b735a5614719f4dc19c8b2ae14f6dfca WatchSource:0}: Error finding container 18a577ed031bf284bd15e88adf076b40b735a5614719f4dc19c8b2ae14f6dfca: Status 404 returned error can't find the container with id 18a577ed031bf284bd15e88adf076b40b735a5614719f4dc19c8b2ae14f6dfca
Jul 29 21:29:40.138714 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:40.138695    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:29:40.328454 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:40.328421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" event={"ID":"e36422d9-b6b6-41c5-935a-52c5c1f53fa9","Type":"ContainerStarted","Data":"18a577ed031bf284bd15e88adf076b40b735a5614719f4dc19c8b2ae14f6dfca"}
Jul 29 21:29:41.335164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:41.335127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" event={"ID":"e36422d9-b6b6-41c5-935a-52c5c1f53fa9","Type":"ContainerStarted","Data":"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"}
Jul 29 21:29:41.335668 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:41.335193    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:29:41.354225 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:41.354157    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" podStartSLOduration=1.90485386 podStartE2EDuration="2.354139728s" totalImagesPullingTime="449.285868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:41.353159546 +0000 UTC m=+2531.602397062" watchObservedRunningTime="2026-07-29 21:29:41.354139728 +0000 UTC m=+2531.603377243"
Jul 29 21:29:47.346075 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:29:47.346041    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:30:03.000965 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:03.000928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:03.472032 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:03.471937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:03.770290 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:03.770239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:04.240804 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:04.240714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:07.303646 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:07.303610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:07.479238 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:07.479205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:08.586070 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.586028    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:30:08.589595 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.589574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.592491 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.592471    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3b326f-serving-cert\""
Jul 29 21:30:08.610593 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.610563    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:30:08.638079 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.638046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:08.706218 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.705974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpg8l\" (UniqueName: \"kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.706218 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.706050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.706218 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.706109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.806806 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.806772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpg8l\" (UniqueName: \"kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.806960 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.806825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.806960 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.806864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.807564 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.807505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.809440 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.809410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.819163 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.819135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpg8l\" (UniqueName: \"kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l\") pod \"maas-api-e2e-api-b-3b326f-fbdc9b744-prldf\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:08.867493 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.867415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:08.899632 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:08.899606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:09.047090 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.047067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:30:09.049785 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:30:09.049750    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b817a09_c92e_4992_b0ee_1de8b2417489.slice/crio-e286ca941135b42f780036de3daeec162815da711f9952a677b03ea7c4b0898e WatchSource:0}: Error finding container e286ca941135b42f780036de3daeec162815da711f9952a677b03ea7c4b0898e: Status 404 returned error can't find the container with id e286ca941135b42f780036de3daeec162815da711f9952a677b03ea7c4b0898e
Jul 29 21:30:09.435812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.435783    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:30:09.439281 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.439260    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.442700 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.442683    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-3b326f-dockercfg-mmd6n\""
Jul 29 21:30:09.443251 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.443232    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-3b326f\""
Jul 29 21:30:09.452216 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.452197    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:30:09.479154 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.479121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" event={"ID":"4b817a09-c92e-4992-b0ee-1de8b2417489","Type":"ContainerStarted","Data":"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"}
Jul 29 21:30:09.479350 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.479182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" event={"ID":"4b817a09-c92e-4992-b0ee-1de8b2417489","Type":"ContainerStarted","Data":"e286ca941135b42f780036de3daeec162815da711f9952a677b03ea7c4b0898e"}
Jul 29 21:30:09.479350 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.479211    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:09.503546 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.503489    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" podStartSLOduration=1.1827533749999999 podStartE2EDuration="1.503470761s" totalImagesPullingTime="320.717386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:09.499726902 +0000 UTC m=+2559.748964417" watchObservedRunningTime="2026-07-29 21:30:09.503470761 +0000 UTC m=+2559.752708275"
Jul 29 21:30:09.609020 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.608986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:09.614813 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.614787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.615060 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.615023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgsvp\" (UniqueName: \"kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.710703 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.710616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:09.716082 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.716046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.716245 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.716188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgsvp\" (UniqueName: \"kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.716828 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.716804    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.724961 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.724939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgsvp\" (UniqueName: \"kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp\") pod \"payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.749964 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.749933    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:09.905743 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:09.905715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:30:09.906694 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:30:09.906665    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2207ad9_f1b0_4fe9_8789_f3ac148f18cc.slice/crio-084efeda830342d2a37fe881452e58616b7bbc7a1e9fb0e1d4a6fa95ac60b073 WatchSource:0}: Error finding container 084efeda830342d2a37fe881452e58616b7bbc7a1e9fb0e1d4a6fa95ac60b073: Status 404 returned error can't find the container with id 084efeda830342d2a37fe881452e58616b7bbc7a1e9fb0e1d4a6fa95ac60b073
Jul 29 21:30:10.486075 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:10.486041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" event={"ID":"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc","Type":"ContainerStarted","Data":"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"}
Jul 29 21:30:10.486278 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:10.486108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" event={"ID":"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc","Type":"ContainerStarted","Data":"084efeda830342d2a37fe881452e58616b7bbc7a1e9fb0e1d4a6fa95ac60b073"}
Jul 29 21:30:10.486402 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:10.486367    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:10.503207 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:10.503147    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" podStartSLOduration=1.503131156 podStartE2EDuration="1.503131156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:10.502298049 +0000 UTC m=+2560.751535566" watchObservedRunningTime="2026-07-29 21:30:10.503131156 +0000 UTC m=+2560.752368673"
Jul 29 21:30:15.492027 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:15.491997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:30:21.494562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:21.494522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:30:32.173533 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:32.173493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:39.009239 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:39.009200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.593138 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.593099    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:30:50.593751 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.593511    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" podUID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" containerName="maas-api" containerID="cri-o://34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315" gracePeriod=30
Jul 29 21:30:50.637464 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.637367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.730095 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.730066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.828259 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.828229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.854374 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.854315    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:30:50.914332 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.914279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.921350 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.921332    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdhk7\" (UniqueName: \"kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7\") pod \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") "
Jul 29 21:30:50.921465 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.921394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") pod \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") "
Jul 29 21:30:50.921465 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.921428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca\") pod \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\" (UID: \"e36422d9-b6b6-41c5-935a-52c5c1f53fa9\") "
Jul 29 21:30:50.921807 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.921787    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e36422d9-b6b6-41c5-935a-52c5c1f53fa9" (UID: "e36422d9-b6b6-41c5-935a-52c5c1f53fa9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:50.923613 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.923595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e36422d9-b6b6-41c5-935a-52c5c1f53fa9" (UID: "e36422d9-b6b6-41c5-935a-52c5c1f53fa9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:30:50.923712 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:50.923693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7" (OuterVolumeSpecName: "kube-api-access-bdhk7") pod "e36422d9-b6b6-41c5-935a-52c5c1f53fa9" (UID: "e36422d9-b6b6-41c5-935a-52c5c1f53fa9"). InnerVolumeSpecName "kube-api-access-bdhk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:51.022653 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.022622    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdhk7\" (UniqueName: \"kubernetes.io/projected/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-kube-api-access-bdhk7\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.022653 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.022649    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.022653 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.022660    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36422d9-b6b6-41c5-935a-52c5c1f53fa9-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.233067 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.232983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.702606 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.702568    2581 generic.go:356] "Generic (PLEG): container finished" podID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" containerID="34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315" exitCode=0
Jul 29 21:30:51.703073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.702622    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"
Jul 29 21:30:51.703073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.702657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" event={"ID":"e36422d9-b6b6-41c5-935a-52c5c1f53fa9","Type":"ContainerDied","Data":"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"}
Jul 29 21:30:51.703073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.702727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm" event={"ID":"e36422d9-b6b6-41c5-935a-52c5c1f53fa9","Type":"ContainerDied","Data":"18a577ed031bf284bd15e88adf076b40b735a5614719f4dc19c8b2ae14f6dfca"}
Jul 29 21:30:51.703073 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.702748    2581 scope.go:117] "RemoveContainer" containerID="34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"
Jul 29 21:30:51.714528 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.714512    2581 scope.go:117] "RemoveContainer" containerID="34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"
Jul 29 21:30:51.714818 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:30:51.714800    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315\": container with ID starting with 34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315 not found: ID does not exist" containerID="34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"
Jul 29 21:30:51.714861 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.714824    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315"} err="failed to get container status \"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315\": rpc error: code = NotFound desc = could not find container \"34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315\": container with ID starting with 34d4df3b660cec07ce0bd04c9737c3feeffb2d808b4b902370418e857ad6f315 not found: ID does not exist"
Jul 29 21:30:51.726663 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.726640    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:30:51.731792 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:51.731764    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e568-694646f547-wh2fm"]
Jul 29 21:30:52.263961 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:52.263927    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" path="/var/lib/kubelet/pods/e36422d9-b6b6-41c5-935a-52c5c1f53fa9/volumes"
Jul 29 21:30:56.996696 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:56.996639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:57.933189 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:57.933148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:58.033556 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:30:58.033519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:13.055514 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.055480    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:31:13.055923 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.055701    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" podUID="4b817a09-c92e-4992-b0ee-1de8b2417489" containerName="maas-api" containerID="cri-o://6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11" gracePeriod=30
Jul 29 21:31:13.073908 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.073883    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:13.113471 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.113443    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:31:13.113670 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.113645    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" podUID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" containerName="payload-pre-processing" containerID="cri-o://329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0" gracePeriod=30
Jul 29 21:31:13.345993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.345960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:13.366650 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.366627    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:31:13.401511 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.401485    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:31:13.430877 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.430846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:13.447980 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.447955    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls\") pod \"4b817a09-c92e-4992-b0ee-1de8b2417489\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") "
Jul 29 21:31:13.448084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.448029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca\") pod \"4b817a09-c92e-4992-b0ee-1de8b2417489\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") "
Jul 29 21:31:13.448084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.448054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpg8l\" (UniqueName: \"kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l\") pod \"4b817a09-c92e-4992-b0ee-1de8b2417489\" (UID: \"4b817a09-c92e-4992-b0ee-1de8b2417489\") "
Jul 29 21:31:13.448407 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.448359    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b817a09-c92e-4992-b0ee-1de8b2417489" (UID: "4b817a09-c92e-4992-b0ee-1de8b2417489"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:31:13.448491 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.448457    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b817a09-c92e-4992-b0ee-1de8b2417489-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:31:13.450204 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.450186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b817a09-c92e-4992-b0ee-1de8b2417489" (UID: "4b817a09-c92e-4992-b0ee-1de8b2417489"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:31:13.450277 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.450216    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l" (OuterVolumeSpecName: "kube-api-access-gpg8l") pod "4b817a09-c92e-4992-b0ee-1de8b2417489" (UID: "4b817a09-c92e-4992-b0ee-1de8b2417489"). InnerVolumeSpecName "kube-api-access-gpg8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:31:13.548785 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.548751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgsvp\" (UniqueName: \"kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp\") pod \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") "
Jul 29 21:31:13.548963 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.548853    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume\") pod \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\" (UID: \"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc\") "
Jul 29 21:31:13.549193 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.549178    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b817a09-c92e-4992-b0ee-1de8b2417489-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:31:13.549255 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.549203    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpg8l\" (UniqueName: \"kubernetes.io/projected/4b817a09-c92e-4992-b0ee-1de8b2417489-kube-api-access-gpg8l\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:31:13.549255 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.549177    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" (UID: "e2207ad9-f1b0-4fe9-8789-f3ac148f18cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:31:13.550954 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.550935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp" (OuterVolumeSpecName: "kube-api-access-hgsvp") pod "e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" (UID: "e2207ad9-f1b0-4fe9-8789-f3ac148f18cc"). InnerVolumeSpecName "kube-api-access-hgsvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:31:13.639940 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.639866    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:13.650121 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.650095    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:31:13.650121 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.650118    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgsvp\" (UniqueName: \"kubernetes.io/projected/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc-kube-api-access-hgsvp\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:31:13.814217 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.814187    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b817a09-c92e-4992-b0ee-1de8b2417489" containerID="6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11" exitCode=0
Jul 29 21:31:13.814420 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.814226    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"
Jul 29 21:31:13.814420 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.814270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" event={"ID":"4b817a09-c92e-4992-b0ee-1de8b2417489","Type":"ContainerDied","Data":"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"}
Jul 29 21:31:13.814420 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.814329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf" event={"ID":"4b817a09-c92e-4992-b0ee-1de8b2417489","Type":"ContainerDied","Data":"e286ca941135b42f780036de3daeec162815da711f9952a677b03ea7c4b0898e"}
Jul 29 21:31:13.814420 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.814358    2581 scope.go:117] "RemoveContainer" containerID="6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"
Jul 29 21:31:13.815652 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.815633    2581 generic.go:356] "Generic (PLEG): container finished" podID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" containerID="329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0" exitCode=0
Jul 29 21:31:13.815745 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.815676    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"
Jul 29 21:31:13.815745 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.815709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" event={"ID":"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc","Type":"ContainerDied","Data":"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"}
Jul 29 21:31:13.815862 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.815747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2" event={"ID":"e2207ad9-f1b0-4fe9-8789-f3ac148f18cc","Type":"ContainerDied","Data":"084efeda830342d2a37fe881452e58616b7bbc7a1e9fb0e1d4a6fa95ac60b073"}
Jul 29 21:31:13.824778 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.824758    2581 scope.go:117] "RemoveContainer" containerID="6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"
Jul 29 21:31:13.825074 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:31:13.825056    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11\": container with ID starting with 6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11 not found: ID does not exist" containerID="6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"
Jul 29 21:31:13.825144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.825079    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11"} err="failed to get container status \"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11\": rpc error: code = NotFound desc = could not find container \"6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11\": container with ID starting with 6d28de1767563bb8d5ea1adb9253b28897c5bf76cceff3acaec99c76f6607b11 not found: ID does not exist"
Jul 29 21:31:13.825144 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.825097    2581 scope.go:117] "RemoveContainer" containerID="329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"
Jul 29 21:31:13.833123 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.833108    2581 scope.go:117] "RemoveContainer" containerID="329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"
Jul 29 21:31:13.833393 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:31:13.833361    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0\": container with ID starting with 329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0 not found: ID does not exist" containerID="329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"
Jul 29 21:31:13.833479 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.833399    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0"} err="failed to get container status \"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0\": rpc error: code = NotFound desc = could not find container \"329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0\": container with ID starting with 329db51ac01e12ab78342241bfc46dbc149b35658300ec51bc7bf2fb4737c9a0 not found: ID does not exist"
Jul 29 21:31:13.836946 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.836918    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:31:13.844601 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.844577    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3b326f-fbdc9b744-prldf"]
Jul 29 21:31:13.854393 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.854359    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:31:13.860344 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:13.860325    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3b326f-56748b74fb-w9kw2"]
Jul 29 21:31:14.263185 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:14.263155    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b817a09-c92e-4992-b0ee-1de8b2417489" path="/var/lib/kubelet/pods/4b817a09-c92e-4992-b0ee-1de8b2417489/volumes"
Jul 29 21:31:14.263568 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:14.263549    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" path="/var/lib/kubelet/pods/e2207ad9-f1b0-4fe9-8789-f3ac148f18cc/volumes"
Jul 29 21:31:19.613402 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:19.613340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:19.715505 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:19.715474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:19.902181 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:19.902105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:20.180044 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:20.179964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:20.524537 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:20.524507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:20.847087 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:20.846992    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:21.154120 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:21.154026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:23.678856 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:23.678823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:23.781615 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:23.781570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:24.846540 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.846501    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:31:24.847122 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847106    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" containerName="payload-pre-processing"
Jul 29 21:31:24.847194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847124    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" containerName="payload-pre-processing"
Jul 29 21:31:24.847194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847144    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" containerName="maas-api"
Jul 29 21:31:24.847194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847152    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" containerName="maas-api"
Jul 29 21:31:24.847194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847170    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b817a09-c92e-4992-b0ee-1de8b2417489" containerName="maas-api"
Jul 29 21:31:24.847194 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847178    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b817a09-c92e-4992-b0ee-1de8b2417489" containerName="maas-api"
Jul 29 21:31:24.847501 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847255    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36422d9-b6b6-41c5-935a-52c5c1f53fa9" containerName="maas-api"
Jul 29 21:31:24.847501 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847266    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b817a09-c92e-4992-b0ee-1de8b2417489" containerName="maas-api"
Jul 29 21:31:24.847501 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.847280    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2207ad9-f1b0-4fe9-8789-f3ac148f18cc" containerName="payload-pre-processing"
Jul 29 21:31:24.852442 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.852419    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.855319 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.855121    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-eb3087-serving-cert\""
Jul 29 21:31:24.858057 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.857997    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:31:24.861009 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.860984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.861643 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.861233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.861643 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.861332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxn2t\" (UniqueName: \"kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.879495 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.879453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:24.962350 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.962315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxn2t\" (UniqueName: \"kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.963062 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.963037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.963425 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.963369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.963539 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:31:24.963263    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-serving-cert: secret "maas-api-e2e-shared-a-eb3087-serving-cert" not found
Jul 29 21:31:24.963614 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:31:24.963598    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls podName:a807d4a5-422a-4b49-a8b5-6c6ba37aa846 nodeName:}" failed. No retries permitted until 2026-07-29 21:31:25.463572767 +0000 UTC m=+2635.712810274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls") pod "maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" (UID: "a807d4a5-422a-4b49-a8b5-6c6ba37aa846") : secret "maas-api-e2e-shared-a-eb3087-serving-cert" not found
Jul 29 21:31:24.963991 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.963972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:24.974947 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:24.974923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxn2t\" (UniqueName: \"kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:25.049055 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.048998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:25.350094 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.350065    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:25.445220 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.445191    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:25.467478 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.467443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:25.470025 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.470003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") pod \"maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:25.768005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.767970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:25.912597 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:25.912568    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:31:25.915787 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:31:25.915750    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda807d4a5_422a_4b49_a8b5_6c6ba37aa846.slice/crio-fc0c61e57e84c15f8878ecc728277d60dd4ae1f5952c662f79c6f5f598509ab8 WatchSource:0}: Error finding container fc0c61e57e84c15f8878ecc728277d60dd4ae1f5952c662f79c6f5f598509ab8: Status 404 returned error can't find the container with id fc0c61e57e84c15f8878ecc728277d60dd4ae1f5952c662f79c6f5f598509ab8
Jul 29 21:31:26.892343 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:26.892306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" event={"ID":"a807d4a5-422a-4b49-a8b5-6c6ba37aa846","Type":"ContainerStarted","Data":"fc0c61e57e84c15f8878ecc728277d60dd4ae1f5952c662f79c6f5f598509ab8"}
Jul 29 21:31:27.899108 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:27.899070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" event={"ID":"a807d4a5-422a-4b49-a8b5-6c6ba37aa846","Type":"ContainerStarted","Data":"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"}
Jul 29 21:31:27.899649 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:27.899186    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:27.917303 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:27.917247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" podStartSLOduration=2.8855849300000003 podStartE2EDuration="3.917231391s" totalImagesPullingTime="1.031646461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:31:27.916335563 +0000 UTC m=+2638.165573082" watchObservedRunningTime="2026-07-29 21:31:27.917231391 +0000 UTC m=+2638.166468905"
Jul 29 21:31:33.909950 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:33.909921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:31:48.677528 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:48.677486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:48.996936 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:48.996896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:49.305707 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:49.305622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:49.605877 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:49.605789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:51.807027 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:51.806995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:51.881608 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:51.881581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:52.840043 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.840009    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:31:52.843808 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.843790    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:52.845892 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.845874    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-46bb38-serving-cert\""
Jul 29 21:31:52.853874 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.853166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:31:52.875030 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.875001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:52.929594 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.929564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:52.929779 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.929609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdncn\" (UniqueName: \"kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:52.929835 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:52.929770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.031408 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.031345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.031588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.031454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.031588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.031486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdncn\" (UniqueName: \"kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.032044 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.032015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.034000 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.033980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.039290 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.039263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdncn\" (UniqueName: \"kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn\") pod \"maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.052872 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.052846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:53.158796 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.158712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:53.303629 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.303606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:31:53.306479 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:31:53.306449    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d61b5e_13f9_4223_9089_5d692ca7ea6a.slice/crio-fdf7c218f8bfc6ac6144fb5297a3866a5e7f4b425714bf11ba63605c6c8ef2d0 WatchSource:0}: Error finding container fdf7c218f8bfc6ac6144fb5297a3866a5e7f4b425714bf11ba63605c6c8ef2d0: Status 404 returned error can't find the container with id fdf7c218f8bfc6ac6144fb5297a3866a5e7f4b425714bf11ba63605c6c8ef2d0
Jul 29 21:31:53.381769 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.381732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:53.482159 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:53.482077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:54.030885 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:54.030855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" event={"ID":"e4d61b5e-13f9-4223-9089-5d692ca7ea6a","Type":"ContainerStarted","Data":"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"}
Jul 29 21:31:54.031214 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:54.030906    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" event={"ID":"e4d61b5e-13f9-4223-9089-5d692ca7ea6a","Type":"ContainerStarted","Data":"fdf7c218f8bfc6ac6144fb5297a3866a5e7f4b425714bf11ba63605c6c8ef2d0"}
Jul 29 21:31:54.031214 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:54.030936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:31:54.050768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:31:54.050727    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" podStartSLOduration=1.4046549289999999 podStartE2EDuration="2.050713481s" totalImagesPullingTime="646.058552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:31:54.049203035 +0000 UTC m=+2664.298440550" watchObservedRunningTime="2026-07-29 21:31:54.050713481 +0000 UTC m=+2664.299950996"
Jul 29 21:32:00.042619 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:32:00.042592    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:32:16.773736 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:32:16.773696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:32:30.824464 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:32:30.824333    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:32:30.833329 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:32:30.833304    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:32:36.948783 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:32:36.948754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:00.914790 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:33:00.914750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:24.871004 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:33:24.870968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:24.969633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:33:24.969604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:02.222681 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:35:02.222584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:03.809971 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:35:03.809942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:05.682695 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:35:05.682661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:37:30.855463 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:37:30.855313    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:37:30.864622 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:37:30.864603    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:37:36.093731 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:37:36.093699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:37:36.953027 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:37:36.952990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:37:56.392549 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:37:56.392514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:38:37.311359 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:38:37.311280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:27.534133 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:27.534088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:27.635974 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:27.635932    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:36.587476 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:36.587432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:37.699694 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:37.699657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:47.687654 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:47.687618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:52.491901 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:39:52.491812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:40:14.294067 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:40:14.294032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:32.969662 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:32.969577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:33.082658 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:33.082619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:51.655917 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:51.655881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:52.063902 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:52.063865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:52.258411 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:52.258368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:55.446977 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:55.446917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:55.584488 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:55.584453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:56.731619 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.731581    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:41:56.735578 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.735557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.738125 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:41:56.738097    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" is forbidden: User \"system:node:ip-10-0-135-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-229.ec2.internal' and this object" podUID="907fc815-9ff7-450b-925c-3be581976a72" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.738711 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:41:56.738685    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-ipp-a-f2f2fe-serving-cert\" is forbidden: User \"system:node:ip-10-0-135-229.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-229.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f2f2fe-serving-cert\"" type="*v1.Secret"
Jul 29 21:41:56.780966 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.780940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.781071 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.781000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.781071 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.781051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlc6z\" (UniqueName: \"kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.795356 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.795316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:41:56.856142 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.856108    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:56.881948 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.881902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.882100 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.881991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.882100 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.882031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc6z\" (UniqueName: \"kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.882624 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.882602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:56.896755 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:56.896725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc6z\" (UniqueName: \"kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:57.070892 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.070860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:57.713333 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.713303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:57.854703 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.854669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:57.882350 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:41:57.882323    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-serving-cert: failed to sync secret cache: timed out waiting for the condition
Jul 29 21:41:57.882505 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:41:57.882413    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls podName:907fc815-9ff7-450b-925c-3be581976a72 nodeName:}" failed. No retries permitted until 2026-07-29 21:41:58.382372394 +0000 UTC m=+3268.631609887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls") pod "maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" (UID: "907fc815-9ff7-450b-925c-3be581976a72") : failed to sync secret cache: timed out waiting for the condition
Jul 29 21:41:57.906025 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.906001    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:41:57.909848 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.909833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:57.921326 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.921307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f2f2fe\""
Jul 29 21:41:57.937953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.937928    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:41:57.969172 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.969103    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:41:57.972280 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.972263    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f2f2fe-serving-cert\""
Jul 29 21:41:57.972742 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.972725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:57.976762 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.976740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f2f2fe-dockercfg-t92gk\""
Jul 29 21:41:57.992787 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.992763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:57.992869 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.992821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrld\" (UniqueName: \"kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:57.992912 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.992872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw7sc\" (UniqueName: \"kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:57.992912 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:57.992897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.014005 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.013985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:41:58.094044 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrld\" (UniqueName: \"kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.094224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw7sc\" (UniqueName: \"kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:58.094224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.094224 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:58.094912 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.095006 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.094972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:58.112654 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.112633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw7sc\" (UniqueName: \"kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc\") pod \"payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:58.113733 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.113714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrld\" (UniqueName: \"kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld\") pod \"payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.220890 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.220803    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:58.283245 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.283212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:58.394569 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.394454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:41:58.396148 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:41:58.396095    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42a98d0_1687_4fce_be20_7996e72b2c0c.slice/crio-e99e341c7f3f50f6cf3300399d8d31a3b70c69fe60df32e8bc1ab4aefd9baae0 WatchSource:0}: Error finding container e99e341c7f3f50f6cf3300399d8d31a3b70c69fe60df32e8bc1ab4aefd9baae0: Status 404 returned error can't find the container with id e99e341c7f3f50f6cf3300399d8d31a3b70c69fe60df32e8bc1ab4aefd9baae0
Jul 29 21:41:58.397758 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.397561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:58.400839 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.400817    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:58.452108 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.452084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:41:58.455559 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:41:58.455534    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c4cf9a_0c7c_4b54_b86d_1ea69a2a1863.slice/crio-dd84bedceac96f4d892b131621e3d8a8e4396fb0bf95db035a7ccb3549dcf9f2 WatchSource:0}: Error finding container dd84bedceac96f4d892b131621e3d8a8e4396fb0bf95db035a7ccb3549dcf9f2: Status 404 returned error can't find the container with id dd84bedceac96f4d892b131621e3d8a8e4396fb0bf95db035a7ccb3549dcf9f2
Jul 29 21:41:58.546054 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.546024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:41:58.722448 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.722425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:41:58.724935 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:41:58.724912    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod907fc815_9ff7_450b_925c_3be581976a72.slice/crio-643eec967c7fdc8e20907698a58d6f072a883d1a3fb93d0d5fb2a5d6db7ee48b WatchSource:0}: Error finding container 643eec967c7fdc8e20907698a58d6f072a883d1a3fb93d0d5fb2a5d6db7ee48b: Status 404 returned error can't find the container with id 643eec967c7fdc8e20907698a58d6f072a883d1a3fb93d0d5fb2a5d6db7ee48b
Jul 29 21:41:58.726410 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:58.726377    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:41:59.074169 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.074129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" event={"ID":"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863","Type":"ContainerStarted","Data":"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"}
Jul 29 21:41:59.074652 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.074196    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:41:59.074652 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.074218    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" event={"ID":"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863","Type":"ContainerStarted","Data":"dd84bedceac96f4d892b131621e3d8a8e4396fb0bf95db035a7ccb3549dcf9f2"}
Jul 29 21:41:59.075403 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.075364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" event={"ID":"907fc815-9ff7-450b-925c-3be581976a72","Type":"ContainerStarted","Data":"643eec967c7fdc8e20907698a58d6f072a883d1a3fb93d0d5fb2a5d6db7ee48b"}
Jul 29 21:41:59.076831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.076807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" event={"ID":"f42a98d0-1687-4fce-be20-7996e72b2c0c","Type":"ContainerStarted","Data":"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"}
Jul 29 21:41:59.076939 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.076859    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:41:59.076939 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.076885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" event={"ID":"f42a98d0-1687-4fce-be20-7996e72b2c0c","Type":"ContainerStarted","Data":"e99e341c7f3f50f6cf3300399d8d31a3b70c69fe60df32e8bc1ab4aefd9baae0"}
Jul 29 21:41:59.118952 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.118903    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" podStartSLOduration=2.118889863 podStartE2EDuration="2.118889863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:41:59.116066709 +0000 UTC m=+3269.365304224" watchObservedRunningTime="2026-07-29 21:41:59.118889863 +0000 UTC m=+3269.368127378"
Jul 29 21:41:59.141009 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:41:59.140952    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" podStartSLOduration=2.140935149 podStartE2EDuration="2.140935149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:41:59.138324803 +0000 UTC m=+3269.387562318" watchObservedRunningTime="2026-07-29 21:41:59.140935149 +0000 UTC m=+3269.390172658"
Jul 29 21:42:00.083532 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:00.083493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" event={"ID":"907fc815-9ff7-450b-925c-3be581976a72","Type":"ContainerStarted","Data":"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"}
Jul 29 21:42:00.084023 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:00.083685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:42:00.107142 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:00.107086    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" podStartSLOduration=3.685942562 podStartE2EDuration="4.107070171s" totalImagesPullingTime="421.127609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:42:00.103672998 +0000 UTC m=+3270.352910515" watchObservedRunningTime="2026-07-29 21:42:00.107070171 +0000 UTC m=+3270.356307686"
Jul 29 21:42:06.095958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:06.095927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:42:10.085425 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:10.085359    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.58:9004: connect: connection refused"
Jul 29 21:42:10.085943 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:10.085924    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:42:20.084680 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:20.084650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:42:20.468524 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:20.468434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:21.182074 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:21.182033    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:21.608631 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:21.608584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:21.815346 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:21.815301    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:30.886864 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:30.886745    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:42:30.898017 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:30.897998    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:42:35.363011 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:35.362969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:35.490768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:35.490735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:38.076055 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.076008    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:42:38.080556 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.080536    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.083367 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.083349    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-00731e-serving-cert\""
Jul 29 21:42:38.094608 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.094585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:42:38.129651 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.129620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:38.200799 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.200763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.200995 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.200914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.200995 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.200961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jczv\" (UniqueName: \"kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.302424 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.302370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.302596 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:42:38.302508    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-serving-cert: secret "maas-api-e2e-ipp-b-00731e-serving-cert" not found
Jul 29 21:42:38.302596 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:42:38.302562    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls podName:fd1ec29c-ef7b-4b52-940b-0c25e2e36974 nodeName:}" failed. No retries permitted until 2026-07-29 21:42:38.802547627 +0000 UTC m=+3309.051785123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls") pod "maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" (UID: "fd1ec29c-ef7b-4b52-940b-0c25e2e36974") : secret "maas-api-e2e-ipp-b-00731e-serving-cert" not found
Jul 29 21:42:38.302596 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.302511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.302752 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.302629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jczv\" (UniqueName: \"kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.303213 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.303192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.313952 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.313921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jczv\" (UniqueName: \"kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.509658 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.509605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:38.796940 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.796856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:38.808631 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.808597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.811221 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.811187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") pod \"maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:38.991084 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:38.991049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:39.001348 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.001321    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:39.160183 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.160149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:42:39.161965 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:42:39.161941    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1ec29c_ef7b_4b52_940b_0c25e2e36974.slice/crio-1e20e55b42e2e7e10764279e7e34f0f11d7f0abaf4db2cd26bb643c09e2e9277 WatchSource:0}: Error finding container 1e20e55b42e2e7e10764279e7e34f0f11d7f0abaf4db2cd26bb643c09e2e9277: Status 404 returned error can't find the container with id 1e20e55b42e2e7e10764279e7e34f0f11d7f0abaf4db2cd26bb643c09e2e9277
Jul 29 21:42:39.263541 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.263504    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:42:39.268434 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.268412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.271008 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.270989    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-00731e-dockercfg-dqdzk\""
Jul 29 21:42:39.271851 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.271834    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-00731e\""
Jul 29 21:42:39.287518 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.287486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" event={"ID":"fd1ec29c-ef7b-4b52-940b-0c25e2e36974","Type":"ContainerStarted","Data":"1e20e55b42e2e7e10764279e7e34f0f11d7f0abaf4db2cd26bb643c09e2e9277"}
Jul 29 21:42:39.313521 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.313470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8djr4\" (UniqueName: \"kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.313720 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.313681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.316712 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.316686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:42:39.415208 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.415121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.415417 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.415212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8djr4\" (UniqueName: \"kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.415893 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.415871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.439535 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.439503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8djr4\" (UniqueName: \"kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4\") pod \"payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.579811 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.579779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:39.776028 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:39.776003    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:42:39.778045 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:42:39.778016    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bc493f_381a_4c6c_b1d1_0e04732c3813.slice/crio-482b8154083c4f938f756969353cae36939c809573d208b33521e64afc285010 WatchSource:0}: Error finding container 482b8154083c4f938f756969353cae36939c809573d208b33521e64afc285010: Status 404 returned error can't find the container with id 482b8154083c4f938f756969353cae36939c809573d208b33521e64afc285010
Jul 29 21:42:40.295817 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.295776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" event={"ID":"fd1ec29c-ef7b-4b52-940b-0c25e2e36974","Type":"ContainerStarted","Data":"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"}
Jul 29 21:42:40.296274 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.295852    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:40.297376 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.297346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" event={"ID":"72bc493f-381a-4c6c-b1d1-0e04732c3813","Type":"ContainerStarted","Data":"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"}
Jul 29 21:42:40.297545 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.297417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:42:40.297545 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.297437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" event={"ID":"72bc493f-381a-4c6c-b1d1-0e04732c3813","Type":"ContainerStarted","Data":"482b8154083c4f938f756969353cae36939c809573d208b33521e64afc285010"}
Jul 29 21:42:40.324965 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.324920    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" podStartSLOduration=1.863098315 podStartE2EDuration="2.32490671s" totalImagesPullingTime="461.808395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:42:40.323809176 +0000 UTC m=+3310.573046703" watchObservedRunningTime="2026-07-29 21:42:40.32490671 +0000 UTC m=+3310.574144224"
Jul 29 21:42:40.353950 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:40.353904    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" podStartSLOduration=1.353889253 podStartE2EDuration="1.353889253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:42:40.353692187 +0000 UTC m=+3310.602929701" watchObservedRunningTime="2026-07-29 21:42:40.353889253 +0000 UTC m=+3310.603126767"
Jul 29 21:42:46.308109 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:46.308076    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:42:49.125697 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:49.125649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:49.977512 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:49.977446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:42:51.305100 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:42:51.305026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:43:02.009831 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:43:02.009799    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:43:22.889413 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:43:22.889362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:04.516993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:04.516960    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w_b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863/payload-processing/0.log"
Jul 29 21:44:04.640664 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:04.640635    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:44:05.032012 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:05.031977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:05.416226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:05.416138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:05.966337 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:05.966299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:08.592269 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:08.592234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:08.681538 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:08.681503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:11.735103 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.735067    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:11.739630 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.739612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.741767 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.741744    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-50783d-serving-cert\""
Jul 29 21:44:11.748972 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.748947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:11.779289 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.779249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:11.847335 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.847295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.847522 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.847358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-957xw\" (UniqueName: \"kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.847522 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.847456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.887319 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.887285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:11.948666 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.948634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.948822 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.948679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-957xw\" (UniqueName: \"kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.948822 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.948724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.949436 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.949418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.951076 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.951060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:11.957768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:11.957745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-957xw\" (UniqueName: \"kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw\") pod \"maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:12.055168 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.055129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:12.210132 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.210100    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:12.214193 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:44:12.214169    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4732904_054c_471a_96e0_34ba759699be.slice/crio-1b9a0ebff29bdc65a003f0ff08410374adfb15fc75afd6e7a0f841d176d116ec WatchSource:0}: Error finding container 1b9a0ebff29bdc65a003f0ff08410374adfb15fc75afd6e7a0f841d176d116ec: Status 404 returned error can't find the container with id 1b9a0ebff29bdc65a003f0ff08410374adfb15fc75afd6e7a0f841d176d116ec
Jul 29 21:44:12.330964 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.330872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:12.430791 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.430761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:12.511764 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.511728    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:12.515615 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.515599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.517911 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.517892    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-50783d\""
Jul 29 21:44:12.517995 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.517892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-50783d-dockercfg-m9df8\""
Jul 29 21:44:12.527075 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.527056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:12.553479 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.553439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.553669 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.553491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dh5w\" (UniqueName: \"kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.654338 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.654303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.654486 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.654352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dh5w\" (UniqueName: \"kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.655263 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.655235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.663196 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.663171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dh5w\" (UniqueName: \"kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w\") pod \"payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.788004 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.787966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" event={"ID":"c4732904-054c-471a-96e0-34ba759699be","Type":"ContainerStarted","Data":"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"}
Jul 29 21:44:12.788421 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.788020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" event={"ID":"c4732904-054c-471a-96e0-34ba759699be","Type":"ContainerStarted","Data":"1b9a0ebff29bdc65a003f0ff08410374adfb15fc75afd6e7a0f841d176d116ec"}
Jul 29 21:44:12.788421 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.788057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:12.806933 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.806877    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" podStartSLOduration=1.4004898639999999 podStartE2EDuration="1.806859514s" totalImagesPullingTime="406.36965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:44:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:44:12.804265698 +0000 UTC m=+3403.053503237" watchObservedRunningTime="2026-07-29 21:44:12.806859514 +0000 UTC m=+3403.056097027"
Jul 29 21:44:12.827722 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.827697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:12.971507 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:12.971481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:12.973202 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:44:12.973175    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0276f1d_056d_4d2a_9551_be4915b17efd.slice/crio-3ff7edc0a2a68335943fb84419f278d46bd8d6b0c4a689c4443a79ae38115069 WatchSource:0}: Error finding container 3ff7edc0a2a68335943fb84419f278d46bd8d6b0c4a689c4443a79ae38115069: Status 404 returned error can't find the container with id 3ff7edc0a2a68335943fb84419f278d46bd8d6b0c4a689c4443a79ae38115069
Jul 29 21:44:13.795472 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:13.795428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" event={"ID":"f0276f1d-056d-4d2a-9551-be4915b17efd","Type":"ContainerStarted","Data":"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"}
Jul 29 21:44:13.795980 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:13.795494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" event={"ID":"f0276f1d-056d-4d2a-9551-be4915b17efd","Type":"ContainerStarted","Data":"3ff7edc0a2a68335943fb84419f278d46bd8d6b0c4a689c4443a79ae38115069"}
Jul 29 21:44:13.795980 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:13.795558    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:13.814665 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:13.814620    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" podStartSLOduration=1.8146029559999999 podStartE2EDuration="1.814602956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:44:13.812745999 +0000 UTC m=+3404.061983541" watchObservedRunningTime="2026-07-29 21:44:13.814602956 +0000 UTC m=+3404.063840466"
Jul 29 21:44:18.801672 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:18.801647    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:23.309071 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:23.308989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:24.803569 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:24.803532    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:33.737728 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:33.737694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:33.835795 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:33.835761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:37.223774 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.223741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:37.910558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.910516    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:37.911726 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.910846    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" podUID="c4732904-054c-471a-96e0-34ba759699be" containerName="maas-api" containerID="cri-o://1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53" gracePeriod=30
Jul 29 21:44:37.945114 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.945087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:37.986221 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.986186    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:37.986473 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:37.986444    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" podUID="f0276f1d-056d-4d2a-9551-be4915b17efd" containerName="payload-pre-processing" containerID="cri-o://398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9" gracePeriod=30
Jul 29 21:44:38.042368 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.042332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:38.136696 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.136555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:38.204700 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.204663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:38.257620 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.257576    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:38.285397 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.285357    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:38.402346 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402318    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-957xw\" (UniqueName: \"kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw\") pod \"c4732904-054c-471a-96e0-34ba759699be\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") "
Jul 29 21:44:38.402558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402403    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls\") pod \"c4732904-054c-471a-96e0-34ba759699be\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") "
Jul 29 21:44:38.402558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume\") pod \"f0276f1d-056d-4d2a-9551-be4915b17efd\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") "
Jul 29 21:44:38.402558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca\") pod \"c4732904-054c-471a-96e0-34ba759699be\" (UID: \"c4732904-054c-471a-96e0-34ba759699be\") "
Jul 29 21:44:38.402558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dh5w\" (UniqueName: \"kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w\") pod \"f0276f1d-056d-4d2a-9551-be4915b17efd\" (UID: \"f0276f1d-056d-4d2a-9551-be4915b17efd\") "
Jul 29 21:44:38.402973 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.402941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0276f1d-056d-4d2a-9551-be4915b17efd" (UID: "f0276f1d-056d-4d2a-9551-be4915b17efd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:44:38.403118 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.403074    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4732904-054c-471a-96e0-34ba759699be" (UID: "c4732904-054c-471a-96e0-34ba759699be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:44:38.405466 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.405440    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4732904-054c-471a-96e0-34ba759699be" (UID: "c4732904-054c-471a-96e0-34ba759699be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:44:38.405578 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.405546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w" (OuterVolumeSpecName: "kube-api-access-5dh5w") pod "f0276f1d-056d-4d2a-9551-be4915b17efd" (UID: "f0276f1d-056d-4d2a-9551-be4915b17efd"). InnerVolumeSpecName "kube-api-access-5dh5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:44:38.405623 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.405574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw" (OuterVolumeSpecName: "kube-api-access-957xw") pod "c4732904-054c-471a-96e0-34ba759699be" (UID: "c4732904-054c-471a-96e0-34ba759699be"). InnerVolumeSpecName "kube-api-access-957xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:44:38.468969 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.468870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:38.504371 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.504334    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-957xw\" (UniqueName: \"kubernetes.io/projected/c4732904-054c-471a-96e0-34ba759699be-kube-api-access-957xw\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:44:38.504371 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.504366    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4732904-054c-471a-96e0-34ba759699be-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:44:38.504371 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.504376    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0276f1d-056d-4d2a-9551-be4915b17efd-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:44:38.504633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.504411    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4732904-054c-471a-96e0-34ba759699be-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:44:38.504633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.504420    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dh5w\" (UniqueName: \"kubernetes.io/projected/f0276f1d-056d-4d2a-9551-be4915b17efd-kube-api-access-5dh5w\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:44:38.936431 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.936398    2581 generic.go:356] "Generic (PLEG): container finished" podID="c4732904-054c-471a-96e0-34ba759699be" containerID="1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53" exitCode=0
Jul 29 21:44:38.936588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.936436    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"
Jul 29 21:44:38.936588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.936428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" event={"ID":"c4732904-054c-471a-96e0-34ba759699be","Type":"ContainerDied","Data":"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"}
Jul 29 21:44:38.936588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.936501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9" event={"ID":"c4732904-054c-471a-96e0-34ba759699be","Type":"ContainerDied","Data":"1b9a0ebff29bdc65a003f0ff08410374adfb15fc75afd6e7a0f841d176d116ec"}
Jul 29 21:44:38.936588 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.936529    2581 scope.go:117] "RemoveContainer" containerID="1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"
Jul 29 21:44:38.938016 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.937982    2581 generic.go:356] "Generic (PLEG): container finished" podID="f0276f1d-056d-4d2a-9551-be4915b17efd" containerID="398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9" exitCode=0
Jul 29 21:44:38.938107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.938022    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"
Jul 29 21:44:38.938107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.938042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" event={"ID":"f0276f1d-056d-4d2a-9551-be4915b17efd","Type":"ContainerDied","Data":"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"}
Jul 29 21:44:38.938107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.938098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh" event={"ID":"f0276f1d-056d-4d2a-9551-be4915b17efd","Type":"ContainerDied","Data":"3ff7edc0a2a68335943fb84419f278d46bd8d6b0c4a689c4443a79ae38115069"}
Jul 29 21:44:38.947090 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.947075    2581 scope.go:117] "RemoveContainer" containerID="1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"
Jul 29 21:44:38.947361 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:44:38.947344    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53\": container with ID starting with 1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53 not found: ID does not exist" containerID="1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"
Jul 29 21:44:38.947471 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.947367    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53"} err="failed to get container status \"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53\": rpc error: code = NotFound desc = could not find container \"1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53\": container with ID starting with 1973274eb221d1a915adf5ad9af42fbd973196907b138cea97079b848e1a4e53 not found: ID does not exist"
Jul 29 21:44:38.947471 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.947411    2581 scope.go:117] "RemoveContainer" containerID="398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"
Jul 29 21:44:38.956886 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.956698    2581 scope.go:117] "RemoveContainer" containerID="398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"
Jul 29 21:44:38.957279 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:44:38.957200    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9\": container with ID starting with 398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9 not found: ID does not exist" containerID="398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"
Jul 29 21:44:38.957279 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.957246    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9"} err="failed to get container status \"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9\": rpc error: code = NotFound desc = could not find container \"398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9\": container with ID starting with 398e2ffcbbabd359156b2a963b4c27aed1caf1ca0dc36dd809aaf8203b743ee9 not found: ID does not exist"
Jul 29 21:44:38.960059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.960039    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:38.964212 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.964193    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-50783d-859d49cc54-6dhc9"]
Jul 29 21:44:38.975173 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.975153    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:38.978459 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:38.978426    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-50783d-5c8c79c688-zssdh"]
Jul 29 21:44:40.264920 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:40.264884    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4732904-054c-471a-96e0-34ba759699be" path="/var/lib/kubelet/pods/c4732904-054c-471a-96e0-34ba759699be/volumes"
Jul 29 21:44:40.265439 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:40.265417    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0276f1d-056d-4d2a-9551-be4915b17efd" path="/var/lib/kubelet/pods/f0276f1d-056d-4d2a-9551-be4915b17efd/volumes"
Jul 29 21:44:44.889633 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:44.889598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:44.988913 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:44.988884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:45.170798 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:45.170723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:45.409261 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:44:45.409228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:01.986052 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:01.986016    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:45:01.986578 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:01.986261    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" podUID="907fc815-9ff7-450b-925c-3be581976a72" containerName="maas-api" containerID="cri-o://2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431" gracePeriod=30
Jul 29 21:45:02.037143 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.037100    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.078398 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.078352    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:45:02.078628 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.078603    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerName="payload-pre-processing" containerID="cri-o://1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b" gracePeriod=30
Jul 29 21:45:02.086862 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.086832    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:45:02.087114 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.087058    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" podUID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" containerName="payload-processing" containerID="cri-o://6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251" gracePeriod=30
Jul 29 21:45:02.292536 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.292467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.367103 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.367079    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:45:02.370908 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.370885    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:45:02.391534 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.391502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.413636 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.413608    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:45:02.455136 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") pod \"907fc815-9ff7-450b-925c-3be581976a72\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume\") pod \"f42a98d0-1687-4fce-be20-7996e72b2c0c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455174    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca\") pod \"907fc815-9ff7-450b-925c-3be581976a72\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455202    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlc6z\" (UniqueName: \"kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z\") pod \"907fc815-9ff7-450b-925c-3be581976a72\" (UID: \"907fc815-9ff7-450b-925c-3be581976a72\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume\") pod \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlrld\" (UniqueName: \"kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld\") pod \"f42a98d0-1687-4fce-be20-7996e72b2c0c\" (UID: \"f42a98d0-1687-4fce-be20-7996e72b2c0c\") "
Jul 29 21:45:02.455309 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455300    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw7sc\" (UniqueName: \"kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc\") pod \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\" (UID: \"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863\") "
Jul 29 21:45:02.455658 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "907fc815-9ff7-450b-925c-3be581976a72" (UID: "907fc815-9ff7-450b-925c-3be581976a72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:02.455658 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455641    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f42a98d0-1687-4fce-be20-7996e72b2c0c" (UID: "f42a98d0-1687-4fce-be20-7996e72b2c0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:02.455953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455929    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f42a98d0-1687-4fce-be20-7996e72b2c0c-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.455953 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.455954    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/907fc815-9ff7-450b-925c-3be581976a72-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.456115 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.456095    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" (UID: "b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:02.457496 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.457474    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "907fc815-9ff7-450b-925c-3be581976a72" (UID: "907fc815-9ff7-450b-925c-3be581976a72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:45:02.458063 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.458035    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z" (OuterVolumeSpecName: "kube-api-access-tlc6z") pod "907fc815-9ff7-450b-925c-3be581976a72" (UID: "907fc815-9ff7-450b-925c-3be581976a72"). InnerVolumeSpecName "kube-api-access-tlc6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:02.458139 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.458092    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc" (OuterVolumeSpecName: "kube-api-access-fw7sc") pod "b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" (UID: "b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863"). InnerVolumeSpecName "kube-api-access-fw7sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:02.458178 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.458148    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld" (OuterVolumeSpecName: "kube-api-access-tlrld") pod "f42a98d0-1687-4fce-be20-7996e72b2c0c" (UID: "f42a98d0-1687-4fce-be20-7996e72b2c0c"). InnerVolumeSpecName "kube-api-access-tlrld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:02.557226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.557139    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/907fc815-9ff7-450b-925c-3be581976a72-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.557226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.557169    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlc6z\" (UniqueName: \"kubernetes.io/projected/907fc815-9ff7-450b-925c-3be581976a72-kube-api-access-tlc6z\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.557226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.557179    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.557226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.557188    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlrld\" (UniqueName: \"kubernetes.io/projected/f42a98d0-1687-4fce-be20-7996e72b2c0c-kube-api-access-tlrld\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.557226 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.557196    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw7sc\" (UniqueName: \"kubernetes.io/projected/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863-kube-api-access-fw7sc\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:02.634107 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.634067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.819466 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.819377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.891602 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:02.891561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:03.086253 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.086165    2581 generic.go:356] "Generic (PLEG): container finished" podID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerID="1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b" exitCode=0
Jul 29 21:45:03.086253 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.086219    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"
Jul 29 21:45:03.087106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.086243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" event={"ID":"f42a98d0-1687-4fce-be20-7996e72b2c0c","Type":"ContainerDied","Data":"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"}
Jul 29 21:45:03.087106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.086307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c" event={"ID":"f42a98d0-1687-4fce-be20-7996e72b2c0c","Type":"ContainerDied","Data":"e99e341c7f3f50f6cf3300399d8d31a3b70c69fe60df32e8bc1ab4aefd9baae0"}
Jul 29 21:45:03.087106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.086335    2581 scope.go:117] "RemoveContainer" containerID="1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"
Jul 29 21:45:03.089423 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.089367    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" containerID="6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251" exitCode=0
Jul 29 21:45:03.089594 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.089559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" event={"ID":"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863","Type":"ContainerDied","Data":"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"}
Jul 29 21:45:03.089665 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.089608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w" event={"ID":"b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863","Type":"ContainerDied","Data":"dd84bedceac96f4d892b131621e3d8a8e4396fb0bf95db035a7ccb3549dcf9f2"}
Jul 29 21:45:03.089753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.089674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"
Jul 29 21:45:03.096816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.092880    2581 generic.go:356] "Generic (PLEG): container finished" podID="907fc815-9ff7-450b-925c-3be581976a72" containerID="2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431" exitCode=0
Jul 29 21:45:03.096816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.092953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" event={"ID":"907fc815-9ff7-450b-925c-3be581976a72","Type":"ContainerDied","Data":"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"}
Jul 29 21:45:03.096816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.092991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg" event={"ID":"907fc815-9ff7-450b-925c-3be581976a72","Type":"ContainerDied","Data":"643eec967c7fdc8e20907698a58d6f072a883d1a3fb93d0d5fb2a5d6db7ee48b"}
Jul 29 21:45:03.096816 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.093053    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"
Jul 29 21:45:03.102489 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.102422    2581 scope.go:117] "RemoveContainer" containerID="1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"
Jul 29 21:45:03.102905 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:45:03.102804    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b\": container with ID starting with 1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b not found: ID does not exist" containerID="1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"
Jul 29 21:45:03.103003 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.102894    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b"} err="failed to get container status \"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b\": rpc error: code = NotFound desc = could not find container \"1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b\": container with ID starting with 1a9a2e3fdd1af578eeb3c584be14f6223d8a3fcffa5c793f970b09013958dd1b not found: ID does not exist"
Jul 29 21:45:03.103003 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.102919    2581 scope.go:117] "RemoveContainer" containerID="6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"
Jul 29 21:45:03.112219 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.112202    2581 scope.go:117] "RemoveContainer" containerID="6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"
Jul 29 21:45:03.113517 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:45:03.113486    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251\": container with ID starting with 6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251 not found: ID does not exist" containerID="6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"
Jul 29 21:45:03.113610 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.113521    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251"} err="failed to get container status \"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251\": rpc error: code = NotFound desc = could not find container \"6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251\": container with ID starting with 6fefe8db2dd802373c2f50890934c5c1bdb396b61bd8338bd79f8df182a74251 not found: ID does not exist"
Jul 29 21:45:03.113610 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.113542    2581 scope.go:117] "RemoveContainer" containerID="2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"
Jul 29 21:45:03.115932 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.115912    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:45:03.121907 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.121852    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f2f2fe-7946db8768-2b28c"]
Jul 29 21:45:03.123255 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.123235    2581 scope.go:117] "RemoveContainer" containerID="2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"
Jul 29 21:45:03.123612 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:45:03.123593    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431\": container with ID starting with 2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431 not found: ID does not exist" containerID="2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"
Jul 29 21:45:03.123719 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.123619    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431"} err="failed to get container status \"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431\": rpc error: code = NotFound desc = could not find container \"2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431\": container with ID starting with 2e55b104e00bee43cdb4168bb604d3cb51553e9bd09e61340f8d584175a71431 not found: ID does not exist"
Jul 29 21:45:03.130855 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.130834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:45:03.134429 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.134364    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f2f2fe-695d577775-slxkg"]
Jul 29 21:45:03.144820 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.144797    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:45:03.153695 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:03.153669    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f2f2fe-585c8b8888-vjd9w"]
Jul 29 21:45:04.264249 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:04.264215    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="907fc815-9ff7-450b-925c-3be581976a72" path="/var/lib/kubelet/pods/907fc815-9ff7-450b-925c-3be581976a72/volumes"
Jul 29 21:45:04.264776 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:04.264760    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" path="/var/lib/kubelet/pods/b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863/volumes"
Jul 29 21:45:04.265096 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:04.265084    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" path="/var/lib/kubelet/pods/f42a98d0-1687-4fce-be20-7996e72b2c0c/volumes"
Jul 29 21:45:07.707000 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:07.706969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:44.339947 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:44.339884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:45.696681 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:45.696650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:51.212282 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:51.212206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:51.312591 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:51.312549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:51.484240 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:51.484145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:51.710886 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:51.710795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:58.553000 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.552960    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:45:58.553531 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.553197    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" podUID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" containerName="maas-api" containerID="cri-o://b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078" gracePeriod=30
Jul 29 21:45:58.585440 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.585407    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:58.650571 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.650540    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:45:58.650779 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.650746    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" podUID="72bc493f-381a-4c6c-b1d1-0e04732c3813" containerName="payload-pre-processing" containerID="cri-o://79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052" gracePeriod=30
Jul 29 21:45:58.695024 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.694996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:58.792313 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.792278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:58.826096 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.826035    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:45:58.892167 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.892138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:58.922861 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.922842    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:45:58.926926 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.926898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jczv\" (UniqueName: \"kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv\") pod \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") "
Jul 29 21:45:58.927044 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.926932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") pod \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") "
Jul 29 21:45:58.927044 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.927043    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume\") pod \"72bc493f-381a-4c6c-b1d1-0e04732c3813\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") "
Jul 29 21:45:58.927151 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.927071    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca\") pod \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\" (UID: \"fd1ec29c-ef7b-4b52-940b-0c25e2e36974\") "
Jul 29 21:45:58.927151 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.927093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8djr4\" (UniqueName: \"kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4\") pod \"72bc493f-381a-4c6c-b1d1-0e04732c3813\" (UID: \"72bc493f-381a-4c6c-b1d1-0e04732c3813\") "
Jul 29 21:45:58.927620 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.927486    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd1ec29c-ef7b-4b52-940b-0c25e2e36974" (UID: "fd1ec29c-ef7b-4b52-940b-0c25e2e36974"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:58.927620 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.927525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72bc493f-381a-4c6c-b1d1-0e04732c3813" (UID: "72bc493f-381a-4c6c-b1d1-0e04732c3813"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:58.929221 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.929196    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd1ec29c-ef7b-4b52-940b-0c25e2e36974" (UID: "fd1ec29c-ef7b-4b52-940b-0c25e2e36974"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:45:58.929636 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.929609    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4" (OuterVolumeSpecName: "kube-api-access-8djr4") pod "72bc493f-381a-4c6c-b1d1-0e04732c3813" (UID: "72bc493f-381a-4c6c-b1d1-0e04732c3813"). InnerVolumeSpecName "kube-api-access-8djr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:58.929763 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:58.929739    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv" (OuterVolumeSpecName: "kube-api-access-5jczv") pod "fd1ec29c-ef7b-4b52-940b-0c25e2e36974" (UID: "fd1ec29c-ef7b-4b52-940b-0c25e2e36974"). InnerVolumeSpecName "kube-api-access-5jczv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:59.028311 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.028280    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72bc493f-381a-4c6c-b1d1-0e04732c3813-plugins-config-volume\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:59.028311 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.028305    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:59.028311 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.028314    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8djr4\" (UniqueName: \"kubernetes.io/projected/72bc493f-381a-4c6c-b1d1-0e04732c3813-kube-api-access-8djr4\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:59.028558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.028322    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jczv\" (UniqueName: \"kubernetes.io/projected/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-kube-api-access-5jczv\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:59.028558 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.028333    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd1ec29c-ef7b-4b52-940b-0c25e2e36974-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:45:59.185411 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.185308    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:59.391774 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.391744    2581 generic.go:356] "Generic (PLEG): container finished" podID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" containerID="b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078" exitCode=0
Jul 29 21:45:59.391958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.391788    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"
Jul 29 21:45:59.391958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.391828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" event={"ID":"fd1ec29c-ef7b-4b52-940b-0c25e2e36974","Type":"ContainerDied","Data":"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"}
Jul 29 21:45:59.391958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.391889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv" event={"ID":"fd1ec29c-ef7b-4b52-940b-0c25e2e36974","Type":"ContainerDied","Data":"1e20e55b42e2e7e10764279e7e34f0f11d7f0abaf4db2cd26bb643c09e2e9277"}
Jul 29 21:45:59.391958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.391906    2581 scope.go:117] "RemoveContainer" containerID="b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"
Jul 29 21:45:59.393307 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.393284    2581 generic.go:356] "Generic (PLEG): container finished" podID="72bc493f-381a-4c6c-b1d1-0e04732c3813" containerID="79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052" exitCode=0
Jul 29 21:45:59.393430 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.393376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" event={"ID":"72bc493f-381a-4c6c-b1d1-0e04732c3813","Type":"ContainerDied","Data":"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"}
Jul 29 21:45:59.393430 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.393411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"
Jul 29 21:45:59.393528 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.393435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd" event={"ID":"72bc493f-381a-4c6c-b1d1-0e04732c3813","Type":"ContainerDied","Data":"482b8154083c4f938f756969353cae36939c809573d208b33521e64afc285010"}
Jul 29 21:45:59.401448 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.401407    2581 scope.go:117] "RemoveContainer" containerID="b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"
Jul 29 21:45:59.401734 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:45:59.401713    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078\": container with ID starting with b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078 not found: ID does not exist" containerID="b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"
Jul 29 21:45:59.401820 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.401746    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078"} err="failed to get container status \"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078\": rpc error: code = NotFound desc = could not find container \"b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078\": container with ID starting with b6e5068e2c160e62dda60d833e482e3fb0bff99bcbbe444dd52ae7c4653af078 not found: ID does not exist"
Jul 29 21:45:59.401820 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.401768    2581 scope.go:117] "RemoveContainer" containerID="79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"
Jul 29 21:45:59.409890 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.409875    2581 scope.go:117] "RemoveContainer" containerID="79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"
Jul 29 21:45:59.410136 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:45:59.410118    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052\": container with ID starting with 79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052 not found: ID does not exist" containerID="79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"
Jul 29 21:45:59.410180 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.410143    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052"} err="failed to get container status \"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052\": rpc error: code = NotFound desc = could not find container \"79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052\": container with ID starting with 79331e930c95e129a836ba19e15f27f633f17d8812cf6a78243e943cada79052 not found: ID does not exist"
Jul 29 21:45:59.425422 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.423956    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:45:59.431187 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.431153    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-00731e-74bd55cd8f-2rxnv"]
Jul 29 21:45:59.451527 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.451434    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:45:59.453753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:45:59.453730    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-00731e-865bcf8f56-vp6hd"]
Jul 29 21:46:00.264458 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:00.264418    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bc493f-381a-4c6c-b1d1-0e04732c3813" path="/var/lib/kubelet/pods/72bc493f-381a-4c6c-b1d1-0e04732c3813/volumes"
Jul 29 21:46:00.264975 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:00.264928    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" path="/var/lib/kubelet/pods/fd1ec29c-ef7b-4b52-940b-0c25e2e36974/volumes"
Jul 29 21:46:05.762557 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:05.762515    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:05.868498 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:05.868400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:06.067295 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:06.067202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:06.294293 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:06.294261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:12.172164 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.172130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:46:12.172955 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.172920    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" podUID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" containerName="maas-api" containerID="cri-o://b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af" gracePeriod=30
Jul 29 21:46:12.206097 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.206071    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:12.300264 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.300228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:12.399719 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.399633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:12.428730 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.428673    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:46:12.463812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.463775    2581 generic.go:356] "Generic (PLEG): container finished" podID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" containerID="b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af" exitCode=0
Jul 29 21:46:12.463993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.463831    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"
Jul 29 21:46:12.463993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.463831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" event={"ID":"a807d4a5-422a-4b49-a8b5-6c6ba37aa846","Type":"ContainerDied","Data":"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"}
Jul 29 21:46:12.463993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.463895    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd" event={"ID":"a807d4a5-422a-4b49-a8b5-6c6ba37aa846","Type":"ContainerDied","Data":"fc0c61e57e84c15f8878ecc728277d60dd4ae1f5952c662f79c6f5f598509ab8"}
Jul 29 21:46:12.463993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.463928    2581 scope.go:117] "RemoveContainer" containerID="b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"
Jul 29 21:46:12.473605 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.473588    2581 scope.go:117] "RemoveContainer" containerID="b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"
Jul 29 21:46:12.473894 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:46:12.473869    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af\": container with ID starting with b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af not found: ID does not exist" containerID="b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"
Jul 29 21:46:12.473941 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.473905    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af"} err="failed to get container status \"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af\": rpc error: code = NotFound desc = could not find container \"b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af\": container with ID starting with b46e942807d2f21ad9623d11e190440f24bb660cd110dadc722d2023a391c5af not found: ID does not exist"
Jul 29 21:46:12.498757 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.498730    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:12.561884 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.561854    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") pod \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") "
Jul 29 21:46:12.561884 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.561888    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxn2t\" (UniqueName: \"kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t\") pod \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") "
Jul 29 21:46:12.562110 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.562027    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca\") pod \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\" (UID: \"a807d4a5-422a-4b49-a8b5-6c6ba37aa846\") "
Jul 29 21:46:12.562355 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.562331    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a807d4a5-422a-4b49-a8b5-6c6ba37aa846" (UID: "a807d4a5-422a-4b49-a8b5-6c6ba37aa846"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:46:12.562500 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.562480    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:12.564141 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.564119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a807d4a5-422a-4b49-a8b5-6c6ba37aa846" (UID: "a807d4a5-422a-4b49-a8b5-6c6ba37aa846"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:46:12.564255 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.564191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t" (OuterVolumeSpecName: "kube-api-access-hxn2t") pod "a807d4a5-422a-4b49-a8b5-6c6ba37aa846" (UID: "a807d4a5-422a-4b49-a8b5-6c6ba37aa846"). InnerVolumeSpecName "kube-api-access-hxn2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:46:12.663958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.663921    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:12.663958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.663952    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxn2t\" (UniqueName: \"kubernetes.io/projected/a807d4a5-422a-4b49-a8b5-6c6ba37aa846-kube-api-access-hxn2t\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:12.788112 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.788085    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:46:12.792791 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.792763    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eb3087-75c86cff5c-cg2zd"]
Jul 29 21:46:12.848799 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:12.848764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:14.263266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:14.263235    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" path="/var/lib/kubelet/pods/a807d4a5-422a-4b49-a8b5-6c6ba37aa846/volumes"
Jul 29 21:46:19.594926 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:19.594889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:19.696299 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:19.696270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:19.889213 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:19.889090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:20.132225 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:20.132183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:25.521768 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.521727    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:46:25.522275 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.521993    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" podUID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" containerName="maas-api" containerID="cri-o://f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3" gracePeriod=30
Jul 29 21:46:25.555514 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.555457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:25.662130 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.662098    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:25.761124 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.761095    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:25.781244 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.781182    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:46:25.842153 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.842115    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:25.896291 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.896263    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls\") pod \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") "
Jul 29 21:46:25.896487 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.896313    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca\") pod \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") "
Jul 29 21:46:25.896487 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.896415    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdncn\" (UniqueName: \"kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn\") pod \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\" (UID: \"e4d61b5e-13f9-4223-9089-5d692ca7ea6a\") "
Jul 29 21:46:25.896779 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.896754    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4d61b5e-13f9-4223-9089-5d692ca7ea6a" (UID: "e4d61b5e-13f9-4223-9089-5d692ca7ea6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:46:25.898600 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.898578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4d61b5e-13f9-4223-9089-5d692ca7ea6a" (UID: "e4d61b5e-13f9-4223-9089-5d692ca7ea6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:46:25.898704 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.898669    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn" (OuterVolumeSpecName: "kube-api-access-zdncn") pod "e4d61b5e-13f9-4223-9089-5d692ca7ea6a" (UID: "e4d61b5e-13f9-4223-9089-5d692ca7ea6a"). InnerVolumeSpecName "kube-api-access-zdncn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:46:25.998106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.998070    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-openshift-service-ca\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:25.998106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.998100    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdncn\" (UniqueName: \"kubernetes.io/projected/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-kube-api-access-zdncn\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:25.998106 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:25.998110    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4d61b5e-13f9-4223-9089-5d692ca7ea6a-maas-api-tls\") on node \"ip-10-0-135-229.ec2.internal\" DevicePath \"\""
Jul 29 21:46:26.131839 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.131756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:26.538065 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.538031    2581 generic.go:356] "Generic (PLEG): container finished" podID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" containerID="f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3" exitCode=0
Jul 29 21:46:26.538488 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.538087    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"
Jul 29 21:46:26.538488 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.538118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" event={"ID":"e4d61b5e-13f9-4223-9089-5d692ca7ea6a","Type":"ContainerDied","Data":"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"}
Jul 29 21:46:26.538488 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.538180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4" event={"ID":"e4d61b5e-13f9-4223-9089-5d692ca7ea6a","Type":"ContainerDied","Data":"fdf7c218f8bfc6ac6144fb5297a3866a5e7f4b425714bf11ba63605c6c8ef2d0"}
Jul 29 21:46:26.538488 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.538204    2581 scope.go:117] "RemoveContainer" containerID="f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"
Jul 29 21:46:26.547734 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.547716    2581 scope.go:117] "RemoveContainer" containerID="f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"
Jul 29 21:46:26.547997 ip-10-0-135-229 kubenswrapper[2581]: E0729 21:46:26.547977    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3\": container with ID starting with f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3 not found: ID does not exist" containerID="f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"
Jul 29 21:46:26.548058 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.548006    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3"} err="failed to get container status \"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3\": rpc error: code = NotFound desc = could not find container \"f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3\": container with ID starting with f95774221caf8cd2cfb86818e2208acba264f777678b752ae186196623ea43e3 not found: ID does not exist"
Jul 29 21:46:26.554604 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.554583    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:46:26.558321 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:26.558301    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-46bb38-65d7f68bb6-k7pz4"]
Jul 29 21:46:28.263703 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:28.263671    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" path="/var/lib/kubelet/pods/e4d61b5e-13f9-4223-9089-5d692ca7ea6a/volumes"
Jul 29 21:46:32.570285 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:32.570255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:32.661021 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:32.660986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:32.827874 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:32.827786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:33.067864 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:33.067828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:39.203610 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:39.203563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-576968bbcc-pgxwx_118d3809-c82a-47df-bb10-797189ae0102/manager/0.log"
Jul 29 21:46:39.820562 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:39.820533    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57645d8769-pwxhj_ad9d1392-ed53-4e83-a4b4-155ff85abaf1/copy-manifests/0.log"
Jul 29 21:46:39.827198 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:39.827176    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57645d8769-pwxhj_ad9d1392-ed53-4e83-a4b4-155ff85abaf1/manager/0.log"
Jul 29 21:46:40.159823 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:40.159748    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5bc489dd8-lms2j_bf232bb7-f5b5-48c7-b913-6f6ef0a8e115/maas-api/0.log"
Jul 29 21:46:40.604396 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:40.604352    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wcn56_c8457d4f-7070-4d34-b1ef-225e5906418a/postgres/0.log"
Jul 29 21:46:43.262839 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:43.262803    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pnhjv_e973c5ca-5b28-4619-8909-4055f1feb70c/discovery/0.log"
Jul 29 21:46:43.750637 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:43.750607    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-mbw4b_9ef1b538-304f-47b1-a41b-8f9297da9cfd/payload-pre-processing/0.log"
Jul 29 21:46:43.874993 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:43.874957    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:46:43.994260 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:43.994229    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5799b9b4f7-h2vnm_3ea4a380-85fe-4581-9a06-e49d7dc8ca61/router/0.log"
Jul 29 21:46:44.343421 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:44.343368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8228f_34331d85-7713-4627-9aaa-f1d515907336/main/0.log"
Jul 29 21:46:44.563572 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:44.563542    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9jjwv_f1d30376-18e4-4dbb-adcb-9cac2d590373/main/0.log"
Jul 29 21:46:44.779305 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:44.779269    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-28555_cbfd2994-2cac-497d-a659-4fb3642d64ee/main/0.log"
Jul 29 21:46:50.392947 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:50.392902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:50.485216 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:50.485181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:53.419659 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:53.419624    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h78gc_ee45771b-8495-4116-981b-9dcf8cffedcd/global-pull-secret-syncer/0.log"
Jul 29 21:46:53.472950 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:53.472917    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-68k2p_97471526-67b0-415f-a32a-fdcda1de72ce/konnectivity-agent/0.log"
Jul 29 21:46:53.590814 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:46:53.590786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-229.ec2.internal_c882f98f11e192b3a9e8390a2cecc279/haproxy/0.log"
Jul 29 21:47:01.403180 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.403145    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/alertmanager/0.log"
Jul 29 21:47:01.441510 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.441425    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/config-reloader/0.log"
Jul 29 21:47:01.488970 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.488937    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/kube-rbac-proxy-web/0.log"
Jul 29 21:47:01.524715 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.524686    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/kube-rbac-proxy/0.log"
Jul 29 21:47:01.568246 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.568214    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/kube-rbac-proxy-metric/0.log"
Jul 29 21:47:01.625502 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.625473    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/prom-label-proxy/0.log"
Jul 29 21:47:01.659466 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.659436    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6da291d9-4d52-4900-b61b-88ba7b397463/init-config-reloader/0.log"
Jul 29 21:47:01.769960 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.769931    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-45c7j_5686bbdc-efb4-4c49-8cd1-1eb9b053e429/kube-state-metrics/0.log"
Jul 29 21:47:01.794004 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.793981    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-45c7j_5686bbdc-efb4-4c49-8cd1-1eb9b053e429/kube-rbac-proxy-main/0.log"
Jul 29 21:47:01.830250 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:01.830220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-45c7j_5686bbdc-efb4-4c49-8cd1-1eb9b053e429/kube-rbac-proxy-self/0.log"
Jul 29 21:47:02.180300 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.180224    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfcd7_b0623358-ed78-467a-a8c8-f394678da67f/node-exporter/0.log"
Jul 29 21:47:02.210012 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.209990    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfcd7_b0623358-ed78-467a-a8c8-f394678da67f/kube-rbac-proxy/0.log"
Jul 29 21:47:02.263429 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.263407    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfcd7_b0623358-ed78-467a-a8c8-f394678da67f/init-textfile/0.log"
Jul 29 21:47:02.309710 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.309681    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hjr57_17496c2d-27e8-4d9d-9b38-66b1d391c818/kube-rbac-proxy-main/0.log"
Jul 29 21:47:02.340314 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.340284    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hjr57_17496c2d-27e8-4d9d-9b38-66b1d391c818/kube-rbac-proxy-self/0.log"
Jul 29 21:47:02.369978 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.369951    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hjr57_17496c2d-27e8-4d9d-9b38-66b1d391c818/openshift-state-metrics/0.log"
Jul 29 21:47:02.416094 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.416064    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/prometheus/0.log"
Jul 29 21:47:02.443634 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.443560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/config-reloader/0.log"
Jul 29 21:47:02.471957 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.471935    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/thanos-sidecar/0.log"
Jul 29 21:47:02.514152 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.514128    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/kube-rbac-proxy-web/0.log"
Jul 29 21:47:02.541865 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.541836    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/kube-rbac-proxy/0.log"
Jul 29 21:47:02.569753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.569730    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/kube-rbac-proxy-thanos/0.log"
Jul 29 21:47:02.601480 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.601455    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5cfd4640-d800-4271-9dc6-acfae561cd01/init-config-reloader/0.log"
Jul 29 21:47:02.641571 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.641545    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-c4vvj_2e82cc2d-10cf-4fdc-84bf-92916e7b58a3/prometheus-operator/0.log"
Jul 29 21:47:02.684775 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.684743    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-c4vvj_2e82cc2d-10cf-4fdc-84bf-92916e7b58a3/kube-rbac-proxy/0.log"
Jul 29 21:47:02.749196 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.749169    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c887d67c5-f8qgc_67501f0d-06ad-4e50-a743-b8e114129050/telemeter-client/0.log"
Jul 29 21:47:02.774272 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.774248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c887d67c5-f8qgc_67501f0d-06ad-4e50-a743-b8e114129050/reload/0.log"
Jul 29 21:47:02.800812 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:02.800784    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c887d67c5-f8qgc_67501f0d-06ad-4e50-a743-b8e114129050/kube-rbac-proxy/0.log"
Jul 29 21:47:03.558079 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"]
Jul 29 21:47:03.558504 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558490    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerName="payload-pre-processing"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558505    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerName="payload-pre-processing"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="907fc815-9ff7-450b-925c-3be581976a72" containerName="maas-api"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558528    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="907fc815-9ff7-450b-925c-3be581976a72" containerName="maas-api"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558534    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4732904-054c-471a-96e0-34ba759699be" containerName="maas-api"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558540    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4732904-054c-471a-96e0-34ba759699be" containerName="maas-api"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558547    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0276f1d-056d-4d2a-9551-be4915b17efd" containerName="payload-pre-processing"
Jul 29 21:47:03.558553 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558551    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0276f1d-056d-4d2a-9551-be4915b17efd" containerName="payload-pre-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558558    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558563    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558573    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" containerName="payload-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558578    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" containerName="payload-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558586    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558590    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558595    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bc493f-381a-4c6c-b1d1-0e04732c3813" containerName="payload-pre-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558601    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bc493f-381a-4c6c-b1d1-0e04732c3813" containerName="payload-pre-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558609    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558614    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558670    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bc493f-381a-4c6c-b1d1-0e04732c3813" containerName="payload-pre-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558678    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a807d4a5-422a-4b49-a8b5-6c6ba37aa846" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558684    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42a98d0-1687-4fce-be20-7996e72b2c0c" containerName="payload-pre-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558690    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4732904-054c-471a-96e0-34ba759699be" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558697    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d61b5e-13f9-4223-9089-5d692ca7ea6a" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558702    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="907fc815-9ff7-450b-925c-3be581976a72" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558708    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1ec29c-ef7b-4b52-940b-0c25e2e36974" containerName="maas-api"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558714    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c4cf9a-0c7c-4b54-b86d-1ea69a2a1863" containerName="payload-processing"
Jul 29 21:47:03.558801 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.558719    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0276f1d-056d-4d2a-9551-be4915b17efd" containerName="payload-pre-processing"
Jul 29 21:47:03.562266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.562246    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.564906 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.564889    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wkbd4\"/\"default-dockercfg-m2tq2\""
Jul 29 21:47:03.565569 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.565553    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wkbd4\"/\"kube-root-ca.crt\""
Jul 29 21:47:03.565657 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.565570    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wkbd4\"/\"openshift-service-ca.crt\""
Jul 29 21:47:03.576631 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.576606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"]
Jul 29 21:47:03.678059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.678025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-lib-modules\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.678059 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.678056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-podres\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.678266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.678079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t95q8\" (UniqueName: \"kubernetes.io/projected/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-kube-api-access-t95q8\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.678266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.678186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-sys\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.678266 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.678222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-proc\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.778911 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.778876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-sys\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.778911 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.778919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-proc\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.778968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-lib-modules\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.778983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-podres\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.779007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t95q8\" (UniqueName: \"kubernetes.io/projected/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-kube-api-access-t95q8\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.779009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-sys\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.779100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-proc\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779155 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.779114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-podres\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.779366 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.779160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-lib-modules\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.787160 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.787135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t95q8\" (UniqueName: \"kubernetes.io/projected/1ac29c4b-0f71-46ed-8452-0bd5fe773cdc-kube-api-access-t95q8\") pod \"perf-node-gather-daemonset-wmrgs\" (UID: \"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc\") " pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:03.874203 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:03.874123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:04.014911 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.014887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"]
Jul 29 21:47:04.017159 ip-10-0-135-229 kubenswrapper[2581]: W0729 21:47:04.017130    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1ac29c4b_0f71_46ed_8452_0bd5fe773cdc.slice/crio-46fb0a679c741574010c2c062c179cdf8a570510d8171344cacf6d0e91ff7e4d WatchSource:0}: Error finding container 46fb0a679c741574010c2c062c179cdf8a570510d8171344cacf6d0e91ff7e4d: Status 404 returned error can't find the container with id 46fb0a679c741574010c2c062c179cdf8a570510d8171344cacf6d0e91ff7e4d
Jul 29 21:47:04.739500 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.739465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs" event={"ID":"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc","Type":"ContainerStarted","Data":"2251927465c3d0613f91f3a037d60e61070fd8a22fb1e4b2bcf1fd9c618f3cd6"}
Jul 29 21:47:04.739915 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.739517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs" event={"ID":"1ac29c4b-0f71-46ed-8452-0bd5fe773cdc","Type":"ContainerStarted","Data":"46fb0a679c741574010c2c062c179cdf8a570510d8171344cacf6d0e91ff7e4d"}
Jul 29 21:47:04.739915 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.739608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:04.758839 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.758795    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs" podStartSLOduration=1.758780497 podStartE2EDuration="1.758780497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:47:04.757485114 +0000 UTC m=+3575.006722629" watchObservedRunningTime="2026-07-29 21:47:04.758780497 +0000 UTC m=+3575.008018047"
Jul 29 21:47:04.856753 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.856726    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/1.log"
Jul 29 21:47:04.860412 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:04.860367    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9v96z_a8b0866d-ea8c-487d-9cec-9b85d83a185e/console-operator/2.log"
Jul 29 21:47:05.986798 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:05.986772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-tlhzp_f5ffb7a0-6bfd-43bf-8f96-6192cfeab75b/volume-data-source-validator/0.log"
Jul 29 21:47:07.038370 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:07.038341    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2fn8c_80a2cb8b-db33-4b5a-84c7-8e18d332a1d5/dns/0.log"
Jul 29 21:47:07.062307 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:07.062286    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2fn8c_80a2cb8b-db33-4b5a-84c7-8e18d332a1d5/kube-rbac-proxy/0.log"
Jul 29 21:47:07.195861 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:07.195833    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dl4bj_f9327713-5426-4824-88bd-2b849ff5997f/dns-node-resolver/0.log"
Jul 29 21:47:07.886814 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:07.886785    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zwpjs_8cbea866-730e-4c2c-b0a2-e16bfb489af8/node-ca/0.log"
Jul 29 21:47:09.013416 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:09.013366    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pnhjv_e973c5ca-5b28-4619-8909-4055f1feb70c/discovery/0.log"
Jul 29 21:47:09.189973 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:09.189943    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-mbw4b_9ef1b538-304f-47b1-a41b-8f9297da9cfd/payload-pre-processing/0.log"
Jul 29 21:47:09.221606 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:09.221582    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:47:09.245256 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:09.245227    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5799b9b4f7-h2vnm_3ea4a380-85fe-4581-9a06-e49d7dc8ca61/router/0.log"
Jul 29 21:47:10.088994 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:10.088963    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w6z99_3cd52a78-0bfa-49ca-93e5-7f285a6136b4/serve-healthcheck-canary/0.log"
Jul 29 21:47:10.755026 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:10.755000    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wkbd4/perf-node-gather-daemonset-wmrgs"
Jul 29 21:47:10.829497 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:10.829469    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p97db_f5787257-a1af-438f-80f8-b503a6463853/kube-rbac-proxy/0.log"
Jul 29 21:47:10.852771 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:10.852746    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p97db_f5787257-a1af-438f-80f8-b503a6463853/exporter/0.log"
Jul 29 21:47:10.876903 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:10.876860    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p97db_f5787257-a1af-438f-80f8-b503a6463853/extractor/0.log"
Jul 29 21:47:13.679958 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:13.679880    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-576968bbcc-pgxwx_118d3809-c82a-47df-bb10-797189ae0102/manager/0.log"
Jul 29 21:47:14.129778 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.129753    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57645d8769-pwxhj_ad9d1392-ed53-4e83-a4b4-155ff85abaf1/manager/0.log"
Jul 29 21:47:14.161308 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.161285    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57645d8769-pwxhj_ad9d1392-ed53-4e83-a4b4-155ff85abaf1/copy-manifests/0.log"
Jul 29 21:47:14.610013 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.609978    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pnhjv_e973c5ca-5b28-4619-8909-4055f1feb70c/discovery/0.log"
Jul 29 21:47:14.751440 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.751414    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-mbw4b_9ef1b538-304f-47b1-a41b-8f9297da9cfd/payload-pre-processing/0.log"
Jul 29 21:47:14.771510 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.771478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:47:14.793023 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:14.792986    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5799b9b4f7-h2vnm_3ea4a380-85fe-4581-9a06-e49d7dc8ca61/router/0.log"
Jul 29 21:47:15.590598 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:15.590560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64fb5f9c89-gwtft_37ab98a1-3774-4d92-89f2-b1f45b0369b9/manager/0.log"
Jul 29 21:47:15.630270 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:15.630240    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-l94c4_199f1e55-68cc-4d10-afe5-9c97456a94bf/openshift-lws-operator/0.log"
Jul 29 21:47:16.460422 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:16.460376    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pnhjv_e973c5ca-5b28-4619-8909-4055f1feb70c/discovery/0.log"
Jul 29 21:47:16.599088 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:16.599059    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-mbw4b_9ef1b538-304f-47b1-a41b-8f9297da9cfd/payload-pre-processing/0.log"
Jul 29 21:47:16.619234 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:16.619162    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lzqhr_403f5856-237a-4378-8de5-3d66da23c397/payload-processing/0.log"
Jul 29 21:47:16.632783 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:16.632761    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5799b9b4f7-h2vnm_3ea4a380-85fe-4581-9a06-e49d7dc8ca61/router/0.log"
Jul 29 21:47:22.442169 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:22.442141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxnjd_1769ec3c-462f-4fb0-86f9-fcecf9756677/kube-multus-additional-cni-plugins/0.log"
Jul 29 21:47:22.471460 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:22.471434    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxnjd_1769ec3c-462f-4fb0-86f9-fcecf9756677/egress-router-binary-copy/0.log"
Jul 29 21:47:22.495948 ip-10-0-135-229 kubenswrapper[2581]: I0729 21:47:22.495920    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxnjd_1769ec3c-462f-4fb0-86f9-fcecf9756677/cni-plugins/0.log"