Jul 22 12:01:19.444122 ip-10-0-132-143 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.936751    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941166    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941179    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941183    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941186    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941191    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941195    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:19.963996 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941198    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941201    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941203    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941206    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941208    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941211    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941214    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941216    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941219    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941221    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941224    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941227    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941229    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941232    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941234    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941237    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941239    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941249    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941254    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:19.966296 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941258    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941262    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941264    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941267    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941271    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941273    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941276    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941279    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941281    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941284    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941287    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941289    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941292    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941295    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941297    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941300    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941302    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941305    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941307    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941310    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:19.975339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941312    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941315    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941318    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941320    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941323    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941326    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941328    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941330    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941333    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941336    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941338    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941341    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941344    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941346    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941349    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941351    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941354    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941358    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941360    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941363    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:19.977566 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941365    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941368    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941370    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941373    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941375    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941378    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941380    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941383    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941385    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941388    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941390    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941393    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941395    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941397    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941400    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941403    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941405    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941408    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941411    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941413    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:19.983079 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941416    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941418    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941420    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941423    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941425    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941428    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941431    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941433    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941437    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941439    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941442    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941445    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941447    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.941450    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942186    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942193    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942196    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942200    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942202    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942205    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:19.986375 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942207    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942210    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942213    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942215    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942218    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942220    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942223    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942228    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942231    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942235    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942238    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942241    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942243    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942246    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942248    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942251    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942253    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942256    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942259    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:19.990308 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942261    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942264    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942267    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942270    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942272    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942275    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942278    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942280    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942282    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942285    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942288    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942292    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942295    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942298    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942300    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942303    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942306    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942309    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942311    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:19.997397 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942314    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942317    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942321    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942324    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942327    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942329    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942332    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942335    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942337    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942340    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942343    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942345    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942348    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942350    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942353    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942356    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942358    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942361    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942364    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942367    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:19.997944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942369    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942372    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942374    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942377    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942380    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942382    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942385    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942387    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942390    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942392    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942396    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942399    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942401    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942403    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942406    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942409    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942411    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942413    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942417    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942420    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:19.998450 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942422    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942425    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942428    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942430    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942432    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942435    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942438    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942440    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942443    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942445    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942448    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942450    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942453    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942455    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942458    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942871    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942878    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942881    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942884    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942888    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:19.998937 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942890    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942893    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942896    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942898    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942901    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942903    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942906    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942909    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942912    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942915    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942917    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942920    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942923    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942925    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942928    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942930    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942933    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942935    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942941    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942944    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:19.999432 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942947    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942949    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942952    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942954    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942957    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942960    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942962    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942964    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942968    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942971    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942976    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942979    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942983    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942986    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942989    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942991    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942994    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942997    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.942999    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:19.999927 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943002    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943005    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943008    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943011    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943013    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943016    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943019    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943022    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943024    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943027    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943030    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943034    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943036    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943039    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943042    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943044    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943047    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943050    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943052    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943055    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:20.000384 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943057    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943060    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943062    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943066    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943069    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943071    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943075    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943079    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943083    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943086    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943089    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943092    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943095    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943097    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943100    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943103    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943106    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943109    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943112    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:20.000909 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943114    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943117    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943119    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943122    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943126    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943129    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943131    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943134    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943136    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943139    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943141    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943144    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943147    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943149    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943152    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943154    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943222    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943229    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943236    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943241    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943245    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 12:01:20.001368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943248    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943253    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943257    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943261    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943264    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943267    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943270    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943274    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943277    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943280    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943284    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943287    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943290    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943294    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943297    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943302    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943305    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943309    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943312    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943315    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943318    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943322    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943325    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943328    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 12:01:20.001979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943331    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943335    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943338    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943343    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943346    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943349    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943352    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943355    2565 flags.go:64] FLAG: --enable-server="true"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943358    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943363    2565 flags.go:64] FLAG: --event-burst="100"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943366    2565 flags.go:64] FLAG: --event-qps="50"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943369    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943372    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943375    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943378    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943381    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943385    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943388    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943391    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943394    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943397    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943400    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943403    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943407    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943410    2565 flags.go:64] FLAG: --feature-gates=""
Jul 22 12:01:20.002574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943414    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943417    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943420    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943423    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943426    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943441    2565 flags.go:64] FLAG: --help="false"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943445    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943449    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943452    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943455    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943460    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943464    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943467    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943470    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943473    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943476    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943478    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943482    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943485    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943488    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943491    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943494    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943497    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943500    2565 flags.go:64] FLAG: --lock-file=""
Jul 22 12:01:20.003286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943504    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943507    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943510    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943516    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943518    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943521    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943525    2565 flags.go:64] FLAG: --logging-format="text"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943528    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943532    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943534    2565 flags.go:64] FLAG: --manifest-url=""
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943537    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943542    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943545    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943549    2565 flags.go:64] FLAG: --max-pods="110"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943552    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943555    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943558    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943561    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943565    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943568    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943571    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943578    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943581    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943584    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 12:01:20.003967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943587    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943590    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943596    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943599    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943602    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943605    2565 flags.go:64] FLAG: --port="10250"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943608    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943611    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0657223341c1f11c7"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943614    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943618    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943621    2565 flags.go:64] FLAG: --register-node="true"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943624    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943628    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943630    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943633    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943637    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943641    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943644    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943647    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943650    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943653    2565 flags.go:64] FLAG: --runonce="false"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943655    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943658    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943661    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943664    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 12:01:20.004623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943667    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943672    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943675    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943678    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943682    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943684    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943687    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943691    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943694    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943697    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943700    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943705    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943708    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943711    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943715    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943718    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943721    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943724    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943727    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943730    2565 flags.go:64] FLAG: --v="2"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943734    2565 flags.go:64] FLAG: --version="false"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943739    2565 flags.go:64] FLAG: --vmodule=""
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943743    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.943746    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943892    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:20.005288 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943896    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943899    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943902    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943905    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943908    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943911    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943914    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943916    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943921    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943925    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943927    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943930    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943932    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943935    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943937    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943940    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943942    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943945    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943947    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943950    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:20.006098 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943953    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943955    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943958    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943960    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943963    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943967    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943969    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943972    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943974    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943980    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943983    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943986    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943988    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943991    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943995    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.943998    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944001    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944003    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944006    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:20.006685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944008    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944012    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944015    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944017    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944020    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944022    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944025    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944027    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944030    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944032    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944035    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944037    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944040    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944043    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944046    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944048    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944051    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944053    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944056    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944059    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:20.007430 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944061    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944064    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944069    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944073    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944076    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944079    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944081    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944084    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944087    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944089    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944092    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944094    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944097    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944100    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944103    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944105    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944108    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944111    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944113    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:20.008200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944115    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944118    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944120    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944123    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944125    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944128    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944130    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944132    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944135    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944138    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944140    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944142    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944145    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944148    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944150    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944154    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944156    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944159    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944161    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:20.008940 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.944164    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.944849    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.951992    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952005    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952019    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952025    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952093    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952100    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952103    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952106    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952109    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952112    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952115    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952118    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952121    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:20.009505 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952124    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952127    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952129    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952132    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952135    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952138    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952141    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952144    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952147    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952150    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952152    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952156    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952159    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952161    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952165    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952168    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952170    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952173    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952175    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952180    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:20.010037 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952183    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952186    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952189    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952192    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952196    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952198    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952201    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952204    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952206    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952209    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952212    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952214    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952216    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952219    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952222    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952224    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952227    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952229    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952232    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952234    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:20.010554 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952237    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952239    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952242    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952244    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952247    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952250    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952253    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952256    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952259    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952261    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952264    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952267    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952269    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952272    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952274    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952277    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952281    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952283    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952286    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:20.011067 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952289    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952291    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952294    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952296    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952299    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952301    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952304    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952306    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952309    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952311    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952314    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952316    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952319    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952321    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952324    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952327    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952331    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952335    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952344    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:20.011530 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952349    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952352    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952355    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952358    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952360    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952363    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952365    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952368    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952371    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952373    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952377    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952381    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952386    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952493    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952498    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:01:20.011992 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952500    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952504    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952506    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952509    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952512    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952514    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952517    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952520    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952522    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952526    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952529    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952531    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952534    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952536    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952539    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952541    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952544    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952547    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952550    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952553    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:01:20.012386 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952555    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952558    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952561    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952563    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952566    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952568    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952571    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952574    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952577    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952580    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952582    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952585    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952587    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952590    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952592    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952595    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952599    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952602    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952606    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:01:20.012944 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952610    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952613    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952616    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952618    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952621    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952624    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952627    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952630    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952633    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952636    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952639    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952644    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952647    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952650    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952652    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952655    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952658    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952660    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952663    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:01:20.013454 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952665    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952668    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952671    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952674    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952676    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952679    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952682    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952684    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952687    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952689    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952691    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952694    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952696    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952699    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952701    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952703    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952706    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952708    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952711    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952713    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:01:20.013959 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952716    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952718    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952721    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952723    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952727    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952730    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952733    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952735    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952738    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952740    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952743    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952745    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952748    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952751    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952754    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952757    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952759    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952762    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:01:20.014514 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:19.952765    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952769    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.952888    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.955022    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.956798    2565 server.go:1019] "Starting client certificate rotation"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.956901    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.956938    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.961352    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.964115    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.978902    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.981422    2565 log.go:25] "Validated CRI v1 image API"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.982234    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 12:01:20.015014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.986696    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 80733780-942f-4351-a5f9-ea45c33e6ea0:/dev/nvme0n1p4 8f049f46-d9da-44be-88d6-e169fa893a8c:/dev/nvme0n1p3]
Jul 22 12:01:20.015396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.986712    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.992668    2565 manager.go:216] Machine: {Timestamp:2026-07-22 12:01:19.991235958 +0000 UTC m=+0.414667003 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098924 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a1b74ce336b55919a3a7d6561edff SystemUUID:ec2a1b74-ce33-6b55-919a-3a7d6561edff BootID:d93d2535-cf9c-4a1e-b3f5-396df87ad2e0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:cb:c9:75:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:cb:c9:75:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:87:f7:2c:cc:ab Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.992780    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.992886    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.996012    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.996249    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.996276    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-143.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 12:01:20.015429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.997122    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.997132    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.997140    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.997155    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:19.998827    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002203    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002339    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002371    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002385    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002407    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002421    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002433    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002918    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.002936    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.005676    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.011140    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.011170    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012490    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012505    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012510    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012516    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012524    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012529    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012539    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012547    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012556    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012561    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012570    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.012575    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.013459    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.013467    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.015076    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-143.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.015113    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 12:01:20.015650 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.015169    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 12:01:20.016604 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.016591    2565 server.go:1268] "Started kubelet"
Jul 22 12:01:20.016719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.016669    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 12:01:20.016773 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.016742    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 12:01:20.016828 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.016778    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 12:01:20.017480 ip-10-0-132-143 systemd[1]: Started Kubernetes Kubelet.
Jul 22 12:01:20.019567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.019550    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 12:01:20.019778 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.019761    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 12:01:20.025171 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.025145    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 12:01:20.028420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.028392    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 12:01:20.028420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.028395    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 12:01:20.028836 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.028654    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.029204 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.029187    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 12:01:20.029301 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.029189    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 12:01:20.029301 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.029220    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 12:01:20.029407 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.029368    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 12:01:20.029407 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.029378    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 12:01:20.030126 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030088    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 12:01:20.030126 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030117    2565 factory.go:55] Registering systemd factory
Jul 22 12:01:20.030126 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030126    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 22 12:01:20.030385 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030341    2565 factory.go:153] Registering CRI-O factory
Jul 22 12:01:20.030385 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030358    2565 factory.go:223] Registration of the crio container factory successfully
Jul 22 12:01:20.030481 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030400    2565 factory.go:103] Registering Raw factory
Jul 22 12:01:20.030481 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.030414    2565 manager.go:1196] Started watching for new ooms in manager
Jul 22 12:01:20.031092 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.031078    2565 manager.go:349] Starting recovery of all containers
Jul 22 12:01:20.032591 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.032562    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 12:01:20.032719 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.032598    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 12:01:20.034152 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.032900    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-143.ec2.internal.18c49ae98053602d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-143.ec2.internal,UID:ip-10-0-132-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-143.ec2.internal,},FirstTimestamp:2026-07-22 12:01:20.016564269 +0000 UTC m=+0.439995315,LastTimestamp:2026-07-22 12:01:20.016564269 +0000 UTC m=+0.439995315,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-143.ec2.internal,}"
Jul 22 12:01:20.047640 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.045967    2565 manager.go:354] Recovery completed
Jul 22 12:01:20.050305 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.050287    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.052520 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.052504    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.052607 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.052533    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.052607 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.052544    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.053069 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.053049    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 12:01:20.053069 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.053065    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 12:01:20.053198 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.053071    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.053198 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.053084    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:01:20.054025 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054014    2565 policy_none.go:49] "None policy: Start"
Jul 22 12:01:20.054060 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054029    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 12:01:20.054060 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054041    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 12:01:20.054675 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.054591    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-143.ec2.internal.18c49ae98278040a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-143.ec2.internal,UID:ip-10-0-132-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-143.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-143.ec2.internal,},FirstTimestamp:2026-07-22 12:01:20.052519946 +0000 UTC m=+0.475950992,LastTimestamp:2026-07-22 12:01:20.052519946 +0000 UTC m=+0.475950992,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-143.ec2.internal,}"
Jul 22 12:01:20.054739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054674    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.054739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054705    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.054739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054719    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.054981 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.054963    2565 policy_none.go:47] "Start"
Jul 22 12:01:20.061446 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.061430    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lg7ps"
Jul 22 12:01:20.068748 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.068679    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-143.ec2.internal.18c49ae982784b1a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-143.ec2.internal,UID:ip-10-0-132-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-143.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-143.ec2.internal,},FirstTimestamp:2026-07-22 12:01:20.052538138 +0000 UTC m=+0.475969184,LastTimestamp:2026-07-22 12:01:20.052538138 +0000 UTC m=+0.475969184,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-143.ec2.internal,}"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.077918    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lg7ps"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088369    2565 manager.go:337] "Starting Device Plugin manager"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.088405    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088416    2565 server.go:87] "Starting device plugin registration server"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088656    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088667    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088762    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088912    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.088922    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.089309    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 12:01:20.091723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.089367    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.126124 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.126093    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 12:01:20.127577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.127550    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 12:01:20.127697 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.127596    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 12:01:20.127697 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.127620    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 12:01:20.127697 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.127658    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 12:01:20.130197 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.130180    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:20.189701 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.189652    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.190698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.190675    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.190768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.190708    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.190768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.190722    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.190768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.190749    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.199062 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.199039    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.199062 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.199064    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-143.ec2.internal\": node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.219703 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.219676    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.227856 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.227838    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"]
Jul 22 12:01:20.227916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.227901    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.228844 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.228827    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.228913 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.228861    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.228913 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.228876    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.229960 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.229945    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.230106 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230081    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.230160 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230114    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.230636 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230620    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.230709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230645    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.230709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230665    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.230709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230674    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.230709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230650    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.230865 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.230721    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.231819 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.231803    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.231892 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.231828    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:20.232436 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.232423    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:20.232495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.232446    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:20.232495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.232459    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:20.251349 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.251326    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-143.ec2.internal\" not found" node="ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.254714 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.254696    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-143.ec2.internal\" not found" node="ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.320467 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.320391    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.330502 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.330466    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.330502 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.330506    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.330667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.330568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfaaa61dc548aab443d124ac4cc0e021-config\") pod \"kube-apiserver-proxy-ip-10-0-132-143.ec2.internal\" (UID: \"bfaaa61dc548aab443d124ac4cc0e021\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.421455 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.421423    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.430727 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430695    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.430847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.430847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfaaa61dc548aab443d124ac4cc0e021-config\") pod \"kube-apiserver-proxy-ip-10-0-132-143.ec2.internal\" (UID: \"bfaaa61dc548aab443d124ac4cc0e021\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.430847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfaaa61dc548aab443d124ac4cc0e021-config\") pod \"kube-apiserver-proxy-ip-10-0-132-143.ec2.internal\" (UID: \"bfaaa61dc548aab443d124ac4cc0e021\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.430847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430840    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.431031 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.430807    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9de48d61f6640eb2713b7ccadbdffbe5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal\" (UID: \"9de48d61f6640eb2713b7ccadbdffbe5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.522187 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.522144    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.554629 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.554602    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.557079 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.557060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:20.622704 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.622480    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.723062 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.723013    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.823582 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.823552    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.923956 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:20.923893    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:20.956312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.956288    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 12:01:20.956476 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:20.956454    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 12:01:21.024220 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:21.024198    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:21.025649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.025635    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:21.082706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.082671    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 11:56:19 +0000 UTC" deadline="2028-04-02 11:05:01.744144606 +0000 UTC"
Jul 22 12:01:21.082706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.082700    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14879h3m40.661447419s"
Jul 22 12:01:21.109951 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.109932    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:01:21.112543 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.112515    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:01:21.112650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.112556    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:01:21.112650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.112568    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:01:21.124989 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:21.124968    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:21.225640 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:21.225554    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:21.291662 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.291631    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:21.326386 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:21.326343    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:21.426716 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:21.426684    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-143.ec2.internal\" not found"
Jul 22 12:01:21.523621 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.523601    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:21.529024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.529003    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:21.530096 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:21.530064    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de48d61f6640eb2713b7ccadbdffbe5.slice/crio-07fca4821ebb3eced62e709185d486c83bdf9f74237efc22b05ff5165f7b8734 WatchSource:0}: Error finding container 07fca4821ebb3eced62e709185d486c83bdf9f74237efc22b05ff5165f7b8734: Status 404 returned error can't find the container with id 07fca4821ebb3eced62e709185d486c83bdf9f74237efc22b05ff5165f7b8734
Jul 22 12:01:21.531143 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:21.531123    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfaaa61dc548aab443d124ac4cc0e021.slice/crio-d7c2283c2d9897e631d6606aba54b465cf1550adddafdeb0df914ab22f9ae29a WatchSource:0}: Error finding container d7c2283c2d9897e631d6606aba54b465cf1550adddafdeb0df914ab22f9ae29a: Status 404 returned error can't find the container with id d7c2283c2d9897e631d6606aba54b465cf1550adddafdeb0df914ab22f9ae29a
Jul 22 12:01:21.535929 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.535907    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:01:21.541074 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.541057    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:01:21.542030 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.542012    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal"
Jul 22 12:01:21.560412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.560391    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:01:21.788829 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:21.788741    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:01:22.004058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.004029    2565 apiserver.go:52] "Watching apiserver"
Jul 22 12:01:22.011308 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.011278    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 12:01:22.012988 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.012960    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mjqtp","kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69","openshift-cluster-node-tuning-operator/tuned-cthqd","openshift-multus/network-metrics-daemon-kfm2l","openshift-network-operator/iptables-alerter-mlr29","kube-system/konnectivity-agent-bkbtq","openshift-dns/node-resolver-xmpph","openshift-image-registry/node-ca-58ss7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal","openshift-multus/multus-5n6h7","openshift-multus/multus-additional-cni-plugins-rwrfg","openshift-network-diagnostics/network-check-target-6ww7v","openshift-ovn-kubernetes/ovnkube-node-fcthx"]
Jul 22 12:01:22.014304 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.014282    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.016386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.016179    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:22.016386 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.016242    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:22.016386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.016351    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 12:01:22.016386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.016362    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.016631 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.016362    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.016631 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.016478    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7c9bk\""
Jul 22 12:01:22.017316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.017289    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.018891 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.018874    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.019172 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019157    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vbzpc\""
Jul 22 12:01:22.019319 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019299    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 12:01:22.019386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019360    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.019604 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019467    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 12:01:22.019604 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019543    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 12:01:22.019604 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.019600    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.020386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.020369    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.021147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.020931    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.021332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.021308    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.021810 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.021677    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2gl6f\""
Jul 22 12:01:22.022706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.022246    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 12:01:22.022706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.022306    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rrgw5\""
Jul 22 12:01:22.022706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.022604    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.024449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.024427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.025058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.025015    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mmqng\""
Jul 22 12:01:22.025347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.025161    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.025347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.025227    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.025472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.025233    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 12:01:22.025633 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.025613    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.026500 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.026447    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 12:01:22.026587 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.026534    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 12:01:22.026868 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.026849    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kk7cg\""
Jul 22 12:01:22.027516 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.027323    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.027516 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.027389    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:22.027799 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.027766    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.027916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.027897    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hpjpx\""
Jul 22 12:01:22.028179 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.028165    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.028666 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.028651    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.028734 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.028695    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:22.029320 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.029252    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 12:01:22.029959 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.029860    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.031599 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.031581    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.031685 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.031589    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.032093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.032078    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 12:01:22.032251 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.032101    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.032434 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.032421    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6qqct\""
Jul 22 12:01:22.033373 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.033357    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 12:01:22.033373 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.033356    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 12:01:22.033650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.033633    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 12:01:22.033862 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.033846    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 12:01:22.034038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.034022    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dhxbd\""
Jul 22 12:01:22.034156 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.034139    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 12:01:22.034223 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.034118    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 12:01:22.036838 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036818    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-lib-modules\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.036930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f119165-7e05-4abd-b029-76446e4ae860-agent-certs\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.036930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cni-binary-copy\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.036930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/807e7e23-00a5-4e02-addf-9264b282db03-host\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.036930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036925    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036947    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-run\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnfv2\" (UniqueName: \"kubernetes.io/projected/8edb1ee9-df00-4ad9-b693-af21cdfac67e-kube-api-access-xnfv2\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.036994    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-system-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.037125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-device-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037109    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cnibin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037184    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f119165-7e05-4abd-b029-76446e4ae860-konnectivity-ca\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r568\" (UniqueName: \"kubernetes.io/projected/e9c148ea-0e1b-429c-b153-43ca865d9ad6-kube-api-access-6r568\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwdxw\" (UniqueName: \"kubernetes.io/projected/91ebfa0e-768f-4f55-8eb9-654f178b4027-kube-api-access-vwdxw\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-system-cni-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037304    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-host\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-kubernetes\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-conf\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037397    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-multus-certs\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037419    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6vrn\" (UniqueName: \"kubernetes.io/projected/22f07980-325d-4c0e-af77-a09fa01d6ec1-kube-api-access-q6vrn\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.037472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037454    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7vg8\" (UniqueName: \"kubernetes.io/projected/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-kube-api-access-x7vg8\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037502    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-k8s-cni-cncf-io\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-multus\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037569    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-daemon-config\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91ebfa0e-768f-4f55-8eb9-654f178b4027-iptables-alerter-script\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037628    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-hostroot\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksbg\" (UniqueName: \"kubernetes.io/projected/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-kube-api-access-pksbg\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037698    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-binary-copy\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037731    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037763    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-var-lib-kubelet\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86k2b\" (UniqueName: \"kubernetes.io/projected/807e7e23-00a5-4e02-addf-9264b282db03-kube-api-access-86k2b\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037836    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysconfig\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037855    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037878    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-tuned\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037901    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-kubelet-config\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-kubelet\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-etc-kubernetes\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.037988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91ebfa0e-768f-4f55-8eb9-654f178b4027-host-slash\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038010    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zbj4\" (UniqueName: \"kubernetes.io/projected/2481599b-ebae-4b9a-b7f6-90f59129531a-kube-api-access-8zbj4\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-systemd\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038054    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edb1ee9-df00-4ad9-b693-af21cdfac67e-hosts-file\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038074    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-dbus\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038099    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-cnibin\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038159    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edb1ee9-df00-4ad9-b693-af21cdfac67e-tmp-dir\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038196    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-bin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-socket-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.038535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-etc-selinux\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038351    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-os-release\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-socket-dir-parent\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038396    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-netns\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-sys\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/807e7e23-00a5-4e02-addf-9264b282db03-serviceca\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-os-release\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038524    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-modprobe-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038546    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-conf-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038572    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-registration-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038595    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-sys-fs\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.039082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.038622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-tmp\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.039658 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.039484    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:01:22.066132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.066112    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lmwgg"
Jul 22 12:01:22.074740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.074712    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lmwgg"
Jul 22 12:01:22.129893 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.129857    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 12:01:22.131710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.131666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal" event={"ID":"9de48d61f6640eb2713b7ccadbdffbe5","Type":"ContainerStarted","Data":"07fca4821ebb3eced62e709185d486c83bdf9f74237efc22b05ff5165f7b8734"}
Jul 22 12:01:22.132735 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.132712    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal" event={"ID":"bfaaa61dc548aab443d124ac4cc0e021","Type":"ContainerStarted","Data":"d7c2283c2d9897e631d6606aba54b465cf1550adddafdeb0df914ab22f9ae29a"}
Jul 22 12:01:22.138762 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-host\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-kubernetes\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138816    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-conf\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-multus-certs\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6vrn\" (UniqueName: \"kubernetes.io/projected/22f07980-325d-4c0e-af77-a09fa01d6ec1-kube-api-access-q6vrn\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138852    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-kubernetes\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138852    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-host\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.138876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7vg8\" (UniqueName: \"kubernetes.io/projected/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-kube-api-access-x7vg8\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138911    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-multus-certs\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-node-log\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-k8s-cni-cncf-io\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138964    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-multus\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138976    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-conf\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.138990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-daemon-config\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-k8s-cni-cncf-io\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139015    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91ebfa0e-768f-4f55-8eb9-654f178b4027-iptables-alerter-script\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139031    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-multus\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-hostroot\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139095    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksbg\" (UniqueName: \"kubernetes.io/projected/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-kube-api-access-pksbg\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.139188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139143    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-hostroot\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139183    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-binary-copy\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-var-lib-kubelet\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139264    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86k2b\" (UniqueName: \"kubernetes.io/projected/807e7e23-00a5-4e02-addf-9264b282db03-kube-api-access-86k2b\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139289    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-systemd-units\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.139313    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysconfig\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-var-lib-kubelet\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.139370    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:22.639351902 +0000 UTC m=+3.062782946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139315    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysconfig\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139421    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-tuned\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-kubelet-config\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.139776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139467    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-kubelet\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139487    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-etc-kubernetes\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139518    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91ebfa0e-768f-4f55-8eb9-654f178b4027-host-slash\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbj4\" (UniqueName: \"kubernetes.io/projected/2481599b-ebae-4b9a-b7f6-90f59129531a-kube-api-access-8zbj4\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-sysctl-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-systemd\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edb1ee9-df00-4ad9-b693-af21cdfac67e-hosts-file\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-daemon-config\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91ebfa0e-768f-4f55-8eb9-654f178b4027-iptables-alerter-script\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139653    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-dbus\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139663    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-kubelet\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139675    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-cnibin\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139627    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-systemd\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.140380 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91ebfa0e-768f-4f55-8eb9-654f178b4027-host-slash\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-systemd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-etc-kubernetes\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edb1ee9-df00-4ad9-b693-af21cdfac67e-hosts-file\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139742    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-cnibin\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-kubelet-config\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-etc-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139840    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edb1ee9-df00-4ad9-b693-af21cdfac67e-tmp-dir\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139862    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-bin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139871    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f865ab54-07aa-4514-9ce8-64f633eec056-dbus\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-socket-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-var-lib-cni-bin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-etc-selinux\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.139993    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-binary-copy\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.141019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-config\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140056    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-etc-selinux\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140060    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-script-lib\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140058    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-socket-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140061    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140098    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-os-release\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-socket-dir-parent\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-netns\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-slash\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140197    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-os-release\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-host-run-netns\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140209    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-socket-dir-parent\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.141546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140100    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edb1ee9-df00-4ad9-b693-af21cdfac67e-tmp-dir\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-ovn\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140267    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-sys\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140288    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/807e7e23-00a5-4e02-addf-9264b282db03-serviceca\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-os-release\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140363    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-sys\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140359    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140379    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-var-lib-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-os-release\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140415    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-env-overrides\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140447    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-modprobe-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-conf-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-registration-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.142070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-sys-fs\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-tmp\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140533    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-modprobe-d\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140546    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-conf-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140560    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-lib-modules\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140584    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f119165-7e05-4abd-b029-76446e4ae860-agent-certs\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140597    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-sys-fs\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cni-binary-copy\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/807e7e23-00a5-4e02-addf-9264b282db03-host\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140679    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-kubelet\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140706    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-log-socket\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140727    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-netd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-run\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnfv2\" (UniqueName: \"kubernetes.io/projected/8edb1ee9-df00-4ad9-b693-af21cdfac67e-kube-api-access-xnfv2\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.142549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140818    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/807e7e23-00a5-4e02-addf-9264b282db03-serviceca\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140584    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-registration-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140883    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-system-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-run\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140946    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-lib-modules\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/807e7e23-00a5-4e02-addf-9264b282db03-host\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.140821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-system-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141015    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141036    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-device-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141079    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-bin\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141140    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-multus-cni-dir\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.141219    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.141260    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:22.641246178 +0000 UTC m=+3.064677213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:22.143134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141293    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2481599b-ebae-4b9a-b7f6-90f59129531a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141302    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22f07980-325d-4c0e-af77-a09fa01d6ec1-device-dir\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d002808c-1a8b-4831-8e37-d35915f20f6e-ovn-node-metrics-cert\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cnibin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141425    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-netns\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f119165-7e05-4abd-b029-76446e4ae860-konnectivity-ca\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cni-binary-copy\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r568\" (UniqueName: \"kubernetes.io/projected/e9c148ea-0e1b-429c-b153-43ca865d9ad6-kube-api-access-6r568\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwdxw\" (UniqueName: \"kubernetes.io/projected/91ebfa0e-768f-4f55-8eb9-654f178b4027-kube-api-access-vwdxw\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-system-cni-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dvx\" (UniqueName: \"kubernetes.io/projected/d002808c-1a8b-4831-8e37-d35915f20f6e-kube-api-access-54dvx\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141578    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2481599b-ebae-4b9a-b7f6-90f59129531a-system-cni-dir\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.143679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.141686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c148ea-0e1b-429c-b153-43ca865d9ad6-cnibin\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.144093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.142132    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f119165-7e05-4abd-b029-76446e4ae860-konnectivity-ca\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.144093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.143568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-etc-tuned\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.144093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.143598    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-tmp\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.144093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.143763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f119165-7e05-4abd-b029-76446e4ae860-agent-certs\") pod \"konnectivity-agent-bkbtq\" (UID: \"7f119165-7e05-4abd-b029-76446e4ae860\") " pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.151578 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.151233    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:22.151578 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.151266    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:22.151578 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.151278    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:22.151578 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.151327    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:22.651312199 +0000 UTC m=+3.074743247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:22.151578 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.151509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86k2b\" (UniqueName: \"kubernetes.io/projected/807e7e23-00a5-4e02-addf-9264b282db03-kube-api-access-86k2b\") pod \"node-ca-58ss7\" (UID: \"807e7e23-00a5-4e02-addf-9264b282db03\") " pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.151876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.151698    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbj4\" (UniqueName: \"kubernetes.io/projected/2481599b-ebae-4b9a-b7f6-90f59129531a-kube-api-access-8zbj4\") pod \"multus-additional-cni-plugins-rwrfg\" (UID: \"2481599b-ebae-4b9a-b7f6-90f59129531a\") " pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.151876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.151734    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksbg\" (UniqueName: \"kubernetes.io/projected/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-kube-api-access-pksbg\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.151949 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.151873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6vrn\" (UniqueName: \"kubernetes.io/projected/22f07980-325d-4c0e-af77-a09fa01d6ec1-kube-api-access-q6vrn\") pod \"aws-ebs-csi-driver-node-b2x69\" (UID: \"22f07980-325d-4c0e-af77-a09fa01d6ec1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.153318 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.153296    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7vg8\" (UniqueName: \"kubernetes.io/projected/e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba-kube-api-access-x7vg8\") pod \"tuned-cthqd\" (UID: \"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba\") " pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.154136 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.154118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r568\" (UniqueName: \"kubernetes.io/projected/e9c148ea-0e1b-429c-b153-43ca865d9ad6-kube-api-access-6r568\") pod \"multus-5n6h7\" (UID: \"e9c148ea-0e1b-429c-b153-43ca865d9ad6\") " pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.156094 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.156073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnfv2\" (UniqueName: \"kubernetes.io/projected/8edb1ee9-df00-4ad9-b693-af21cdfac67e-kube-api-access-xnfv2\") pod \"node-resolver-xmpph\" (UID: \"8edb1ee9-df00-4ad9-b693-af21cdfac67e\") " pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.157055 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.157035    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwdxw\" (UniqueName: \"kubernetes.io/projected/91ebfa0e-768f-4f55-8eb9-654f178b4027-kube-api-access-vwdxw\") pod \"iptables-alerter-mlr29\" (UID: \"91ebfa0e-768f-4f55-8eb9-654f178b4027\") " pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.242824 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.242824 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-config\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-script-lib\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-slash\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242899    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-ovn\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-var-lib-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242970    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-env-overrides\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-ovn\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-kubelet\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-var-lib-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-log-socket\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243050    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-kubelet\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.242944    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243044    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-netd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243111    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-netd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-bin\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243139    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d002808c-1a8b-4831-8e37-d35915f20f6e-ovn-node-metrics-cert\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-log-socket\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-netns\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54dvx\" (UniqueName: \"kubernetes.io/projected/d002808c-1a8b-4831-8e37-d35915f20f6e-kube-api-access-54dvx\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-cni-bin\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-node-log\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243288    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-systemd-units\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243320    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-systemd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.243553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243341    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-etc-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-etc-openvswitch\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243419    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-env-overrides\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243246    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243480    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-run-systemd\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-node-log\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243514    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-run-netns\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243472    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-host-slash\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-config\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243555    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d002808c-1a8b-4831-8e37-d35915f20f6e-systemd-units\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.244139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.243979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d002808c-1a8b-4831-8e37-d35915f20f6e-ovnkube-script-lib\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.245610 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.245588    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d002808c-1a8b-4831-8e37-d35915f20f6e-ovn-node-metrics-cert\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.254329 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.254308    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dvx\" (UniqueName: \"kubernetes.io/projected/d002808c-1a8b-4831-8e37-d35915f20f6e-kube-api-access-54dvx\") pod \"ovnkube-node-fcthx\" (UID: \"d002808c-1a8b-4831-8e37-d35915f20f6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.326242 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.326158    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-58ss7"
Jul 22 12:01:22.333072 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.333050    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rwrfg"
Jul 22 12:01:22.342342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.342324    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cthqd"
Jul 22 12:01:22.346648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.346628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5n6h7"
Jul 22 12:01:22.352934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.352916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mlr29"
Jul 22 12:01:22.360225 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.360207    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:22.366629 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.366610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmpph"
Jul 22 12:01:22.374032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.374011    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69"
Jul 22 12:01:22.379492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.379472    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:22.594139 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.594108    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f119165_7e05_4abd_b029_76446e4ae860.slice/crio-0236de64f6f9f2ae3ea0477dc468c0032c1369973664d62386729d9c2ea11baa WatchSource:0}: Error finding container 0236de64f6f9f2ae3ea0477dc468c0032c1369973664d62386729d9c2ea11baa: Status 404 returned error can't find the container with id 0236de64f6f9f2ae3ea0477dc468c0032c1369973664d62386729d9c2ea11baa
Jul 22 12:01:22.594989 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.594955    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c148ea_0e1b_429c_b153_43ca865d9ad6.slice/crio-b16407972cac6c64328c01e579acfd4dbcf681e4277f91d6c37da4c09280794a WatchSource:0}: Error finding container b16407972cac6c64328c01e579acfd4dbcf681e4277f91d6c37da4c09280794a: Status 404 returned error can't find the container with id b16407972cac6c64328c01e579acfd4dbcf681e4277f91d6c37da4c09280794a
Jul 22 12:01:22.596529 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.596461    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8edb1ee9_df00_4ad9_b693_af21cdfac67e.slice/crio-9a7f61381340271ff4d7d033b74e675b6e08f25ac4542456dcbf1c745863503e WatchSource:0}: Error finding container 9a7f61381340271ff4d7d033b74e675b6e08f25ac4542456dcbf1c745863503e: Status 404 returned error can't find the container with id 9a7f61381340271ff4d7d033b74e675b6e08f25ac4542456dcbf1c745863503e
Jul 22 12:01:22.597312 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.597286    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ebfa0e_768f_4f55_8eb9_654f178b4027.slice/crio-69b176d61098ccacce19ea65c1a74f8f166b7351ac761ab652c22498c407d359 WatchSource:0}: Error finding container 69b176d61098ccacce19ea65c1a74f8f166b7351ac761ab652c22498c407d359: Status 404 returned error can't find the container with id 69b176d61098ccacce19ea65c1a74f8f166b7351ac761ab652c22498c407d359
Jul 22 12:01:22.600253 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.600231    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c2ef40_07a7_4f00_bd3b_7d96c2d0e2ba.slice/crio-b349a15cf2cf12eb5133757c38d2369a751300b7a8e79999370fc2cee4d9dd57 WatchSource:0}: Error finding container b349a15cf2cf12eb5133757c38d2369a751300b7a8e79999370fc2cee4d9dd57: Status 404 returned error can't find the container with id b349a15cf2cf12eb5133757c38d2369a751300b7a8e79999370fc2cee4d9dd57
Jul 22 12:01:22.601290 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.601073    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd002808c_1a8b_4831_8e37_d35915f20f6e.slice/crio-ed0c20b7a11b2fb1791765af0a63e17c72d190fcd7a53e3cab97aaab44a3a0ba WatchSource:0}: Error finding container ed0c20b7a11b2fb1791765af0a63e17c72d190fcd7a53e3cab97aaab44a3a0ba: Status 404 returned error can't find the container with id ed0c20b7a11b2fb1791765af0a63e17c72d190fcd7a53e3cab97aaab44a3a0ba
Jul 22 12:01:22.602521 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.602048    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f07980_325d_4c0e_af77_a09fa01d6ec1.slice/crio-28e72f5fa5d68d01ac6482be7971f0eb52734a73cb019f1728e76c956fcc1818 WatchSource:0}: Error finding container 28e72f5fa5d68d01ac6482be7971f0eb52734a73cb019f1728e76c956fcc1818: Status 404 returned error can't find the container with id 28e72f5fa5d68d01ac6482be7971f0eb52734a73cb019f1728e76c956fcc1818
Jul 22 12:01:22.603277 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:22.602946    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807e7e23_00a5_4e02_addf_9264b282db03.slice/crio-b107df853da03e1bb5ccad6e7c0d35535446b4b07e6af00a8730ae8118a53be8 WatchSource:0}: Error finding container b107df853da03e1bb5ccad6e7c0d35535446b4b07e6af00a8730ae8118a53be8: Status 404 returned error can't find the container with id b107df853da03e1bb5ccad6e7c0d35535446b4b07e6af00a8730ae8118a53be8
Jul 22 12:01:22.644743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.644709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:22.644936 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.644754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:22.644936 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.644860    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:22.644936 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.644877    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:22.644936 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.644918    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:23.644901474 +0000 UTC m=+4.068332506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:22.644936 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.644933    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:23.644926588 +0000 UTC m=+4.068357619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:22.745839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:22.745806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:22.746008 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.745942    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:22.746008 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.745964    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:22.746008 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.745977    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:22.746102 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:22.746030    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:23.746015634 +0000 UTC m=+4.169446680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:23.078392 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.078290    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:56:21 +0000 UTC" deadline="2028-02-16 02:38:39.119952211 +0000 UTC"
Jul 22 12:01:23.078392 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.078347    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13766h37m16.041608873s"
Jul 22 12:01:23.129245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.128799    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:23.129245 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.128920    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:23.140919 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.140853    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-58ss7" event={"ID":"807e7e23-00a5-4e02-addf-9264b282db03","Type":"ContainerStarted","Data":"b107df853da03e1bb5ccad6e7c0d35535446b4b07e6af00a8730ae8118a53be8"}
Jul 22 12:01:23.145302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.145233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mlr29" event={"ID":"91ebfa0e-768f-4f55-8eb9-654f178b4027","Type":"ContainerStarted","Data":"69b176d61098ccacce19ea65c1a74f8f166b7351ac761ab652c22498c407d359"}
Jul 22 12:01:23.147131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.146963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmpph" event={"ID":"8edb1ee9-df00-4ad9-b693-af21cdfac67e","Type":"ContainerStarted","Data":"9a7f61381340271ff4d7d033b74e675b6e08f25ac4542456dcbf1c745863503e"}
Jul 22 12:01:23.150498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.149852    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5n6h7" event={"ID":"e9c148ea-0e1b-429c-b153-43ca865d9ad6","Type":"ContainerStarted","Data":"b16407972cac6c64328c01e579acfd4dbcf681e4277f91d6c37da4c09280794a"}
Jul 22 12:01:23.155313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.155260    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkbtq" event={"ID":"7f119165-7e05-4abd-b029-76446e4ae860","Type":"ContainerStarted","Data":"0236de64f6f9f2ae3ea0477dc468c0032c1369973664d62386729d9c2ea11baa"}
Jul 22 12:01:23.160466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.160408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerStarted","Data":"462240d3c97cbfe1462c638b92401e36bd2425b00ce51cead17a556ab7bdab43"}
Jul 22 12:01:23.165402 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.165336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69" event={"ID":"22f07980-325d-4c0e-af77-a09fa01d6ec1","Type":"ContainerStarted","Data":"28e72f5fa5d68d01ac6482be7971f0eb52734a73cb019f1728e76c956fcc1818"}
Jul 22 12:01:23.171334 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.168177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"ed0c20b7a11b2fb1791765af0a63e17c72d190fcd7a53e3cab97aaab44a3a0ba"}
Jul 22 12:01:23.173767 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.173711    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cthqd" event={"ID":"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba","Type":"ContainerStarted","Data":"b349a15cf2cf12eb5133757c38d2369a751300b7a8e79999370fc2cee4d9dd57"}
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.654353    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.654435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.654559    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.654616    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:25.654598553 +0000 UTC m=+6.078029586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.654992    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:23.655070 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.655038    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:25.655022885 +0000 UTC m=+6.078453922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:23.755579 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:23.754955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:23.755579 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.755171    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:23.755579 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.755191    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:23.755579 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.755204    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:23.755579 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:23.755256    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:25.755240449 +0000 UTC m=+6.178671485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:24.079431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:24.079295    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:56:21 +0000 UTC" deadline="2028-04-30 15:09:24.627249968 +0000 UTC"
Jul 22 12:01:24.079431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:24.079335    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15555h8m0.547919932s"
Jul 22 12:01:24.130543 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:24.130510    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:24.130701 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:24.130633    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:24.131053 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:24.131033    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:24.131166 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:24.131135    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:25.128524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:25.128449    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:25.129080 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.128570    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:25.187286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:25.187249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal" event={"ID":"bfaaa61dc548aab443d124ac4cc0e021","Type":"ContainerStarted","Data":"b605746cbb543e99ea34f4515901d53c649a4a2d949654efdeda8f2a10aa83d8"}
Jul 22 12:01:25.670986 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:25.670944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:25.671176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:25.671031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:25.671176 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.671153    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:25.671289 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.671209    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:29.671190274 +0000 UTC m=+10.094621319 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:25.671608 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.671587    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:25.671692 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.671641    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:29.671627429 +0000 UTC m=+10.095058466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:25.772239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:25.772203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:25.772424 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.772376    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:25.772424 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.772394    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:25.772424 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.772406    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:25.772587 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:25.772467    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:29.772448115 +0000 UTC m=+10.195879205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:26.128996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:26.128921    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:26.129439 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:26.129050    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:26.129739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:26.129547    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:26.129739 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:26.129659    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:27.128294 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:27.128257    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:27.128485 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:27.128386    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:28.128904 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:28.128870    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:28.129355 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:28.129001    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:28.129355 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:28.129058    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:28.129355 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:28.129170    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:29.129201 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:29.128693    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:29.129201 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.128845    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:29.706546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:29.706499    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:29.706708 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:29.706571    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:29.706708 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.706646    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:29.706708 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.706655    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:29.706708 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.706709    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:37.706691854 +0000 UTC m=+18.130122897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:29.706950 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.706726    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:37.706718183 +0000 UTC m=+18.130149218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:29.807280 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:29.807237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:29.807468 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.807419    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:29.807468 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.807446    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:29.807468 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.807461    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:29.807629 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:29.807526    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:37.807507942 +0000 UTC m=+18.230938978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:30.128850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:30.128758    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:30.129015 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:30.128885    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:30.129095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:30.129029    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:30.129148 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:30.129124    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:31.128650 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:31.128615    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:31.129147 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:31.128751    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:32.128477 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:32.128238    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:32.128657 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:32.128313    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:32.128723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:32.128606    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:32.129067 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:32.128718    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:33.128680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:33.128646    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:33.128875 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:33.128763    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:34.128366 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:34.128334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:34.128557 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:34.128334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:34.128557 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:34.128469    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:34.128691 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:34.128549    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:35.128293 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:35.128248    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:35.128725 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:35.128371    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:36.128854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:36.128813    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:36.128854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:36.128850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:36.129346 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:36.128947    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:36.129346 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:36.129036    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:37.128763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:37.128728    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:37.128972 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.128869    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:37.762040 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:37.762010    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:37.762190 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:37.762056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:37.762225 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.762192    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:37.762258 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.762226    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:37.762288 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.762258    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:53.76224202 +0000 UTC m=+34.185673057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:37.762288 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.762272    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:53.762265534 +0000 UTC m=+34.185696567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:37.862499 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:37.862463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:37.862692 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.862671    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:37.862754 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.862700    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:37.862754 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.862728    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:37.862941 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:37.862810    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:53.862776031 +0000 UTC m=+34.286207078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:38.128109 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:38.128024    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:38.128274 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:38.128039    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:38.128274 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:38.128157    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:38.128274 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:38.128238    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:39.128763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:39.128727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:39.129175 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:39.128888    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:40.129370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:40.129337    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:40.129806 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:40.129428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:40.129806 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:40.129522    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:40.129806 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:40.129662    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:41.128393 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:41.128364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:41.128586 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:41.128481    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:42.129049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.128862    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:42.129558 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:42.129159    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:42.129558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.128870    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:42.129672 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:42.129623    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:42.230239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.230198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"587c77e7bbf2c24534127a38484eadc651fbf622ac218494b1b1227d273bb997"}
Jul 22 12:01:42.230400 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.230344    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"272fa51d724c57440fac601154ee1c4ca8918b7b22858401164061cab352f962"}
Jul 22 12:01:42.230400 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.230389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"18c711a1490b0545d284493a794d4073840c2e74ca40f0f61a5930b79a49afe1"}
Jul 22 12:01:42.231853 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.231824    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cthqd" event={"ID":"e7c2ef40-07a7-4f00-bd3b-7d96c2d0e2ba","Type":"ContainerStarted","Data":"91febc6c5a2f2aac7b452276e165cb42c3c30c93c0c8b0a8983b85973fae3d3e"}
Jul 22 12:01:42.233448 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.233413    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5n6h7" event={"ID":"e9c148ea-0e1b-429c-b153-43ca865d9ad6","Type":"ContainerStarted","Data":"54ff2f846616495934abc602a74977a90cb60ae5acb5d54c611bb4ca1dcd870d"}
Jul 22 12:01:42.259151 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.258999    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-143.ec2.internal" podStartSLOduration=21.258976218 podStartE2EDuration="21.258976218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:25.203506692 +0000 UTC m=+5.626937747" watchObservedRunningTime="2026-07-22 12:01:42.258976218 +0000 UTC m=+22.682407273"
Jul 22 12:01:42.259335 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.259301    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cthqd" podStartSLOduration=3.9897378420000003 podStartE2EDuration="23.259285575s" totalImagesPullingTime="19.269547733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:42.257007013 +0000 UTC m=+22.680438059" watchObservedRunningTime="2026-07-22 12:01:42.259285575 +0000 UTC m=+22.682716629"
Jul 22 12:01:42.282543 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:42.282499    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5n6h7" podStartSLOduration=3.934465333 podStartE2EDuration="23.28248398s" totalImagesPullingTime="19.348018647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:42.281922201 +0000 UTC m=+22.705353253" watchObservedRunningTime="2026-07-22 12:01:42.28248398 +0000 UTC m=+22.705915034"
Jul 22 12:01:43.128461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.128430    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:43.128617 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:43.128535    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:43.237135 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.237105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-58ss7" event={"ID":"807e7e23-00a5-4e02-addf-9264b282db03","Type":"ContainerStarted","Data":"6e6ef4e854f2edc0f1a7b8100eed6d9c868468a4102cdb072d99e75ee86c1ef3"}
Jul 22 12:01:43.238402 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.238381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmpph" event={"ID":"8edb1ee9-df00-4ad9-b693-af21cdfac67e","Type":"ContainerStarted","Data":"ebbe1af285400fa575cd4535a71d31c18f1f77d6439fb3a2968ee75cf380031a"}
Jul 22 12:01:43.239689 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.239669    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkbtq" event={"ID":"7f119165-7e05-4abd-b029-76446e4ae860","Type":"ContainerStarted","Data":"d92d4c061024dc0bcf641ade465adffe973c2dac5ab926667976f33c73a7fd46"}
Jul 22 12:01:43.241113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.241079    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="91d5ec6beaeca5468c66ffd0fa1a931113799c7525aa7e6938285132e849c2a1" exitCode=0
Jul 22 12:01:43.241157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.241146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"91d5ec6beaeca5468c66ffd0fa1a931113799c7525aa7e6938285132e849c2a1"}
Jul 22 12:01:43.242671 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.242646    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69" event={"ID":"22f07980-325d-4c0e-af77-a09fa01d6ec1","Type":"ContainerStarted","Data":"8c2d8bcd766473b06a694b92d8ffdafe96bdfcd01134342cba235d9f62d9d176"}
Jul 22 12:01:43.245771 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.245746    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"4680a3e45172810b260e3c887a75e1690815fd32e42eca07d5366e43256d0d59"}
Jul 22 12:01:43.245871 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.245823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"c87d88000d721e8218006106186de4c170b086e1795c53e50458102331caee7a"}
Jul 22 12:01:43.245871 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.245868    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"fe3dbf2f3435c73d00c5ab26a3e349d96a162f90a8a6e7fcd4c6c81d2af6b940"}
Jul 22 12:01:43.247121 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.247100    2565 generic.go:356] "Generic (PLEG): container finished" podID="9de48d61f6640eb2713b7ccadbdffbe5" containerID="6fac5844d68c3d1fe66e39ce6fe3bfc24f78b3eda5cb6a16d86d97da721547df" exitCode=0
Jul 22 12:01:43.247210 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.247162    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal" event={"ID":"9de48d61f6640eb2713b7ccadbdffbe5","Type":"ContainerDied","Data":"6fac5844d68c3d1fe66e39ce6fe3bfc24f78b3eda5cb6a16d86d97da721547df"}
Jul 22 12:01:43.302295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.302085    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-58ss7" podStartSLOduration=5.037275747 podStartE2EDuration="24.302064322s" totalImagesPullingTime="19.264788575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:43.269228589 +0000 UTC m=+23.692659645" watchObservedRunningTime="2026-07-22 12:01:43.302064322 +0000 UTC m=+23.725495378"
Jul 22 12:01:43.343766 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.343717    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bkbtq" podStartSLOduration=4.068235321 podStartE2EDuration="23.343704265s" totalImagesPullingTime="19.275468944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:43.302548447 +0000 UTC m=+23.725979503" watchObservedRunningTime="2026-07-22 12:01:43.343704265 +0000 UTC m=+23.767135317"
Jul 22 12:01:43.557027 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.556911    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:43.557594 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.557572    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:43.584394 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:43.584340    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xmpph" podStartSLOduration=4.31278041 podStartE2EDuration="23.58432501s" totalImagesPullingTime="19.2715446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:43.404166476 +0000 UTC m=+23.827597532" watchObservedRunningTime="2026-07-22 12:01:43.58432501 +0000 UTC m=+24.007756064"
Jul 22 12:01:44.128379 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.128345    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:44.128723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.128390    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:44.128723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:44.128477    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:44.128723 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:44.128636    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:44.252066 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.252029    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal" event={"ID":"9de48d61f6640eb2713b7ccadbdffbe5","Type":"ContainerStarted","Data":"2affc5a735b756d2c46f3cd3ee8a56720f2e7cadc2b91e710e2b356da5292caf"}
Jul 22 12:01:44.254742 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.254684    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mlr29" event={"ID":"91ebfa0e-768f-4f55-8eb9-654f178b4027","Type":"ContainerStarted","Data":"cd664c98225752c1ee20c2e14c0f88473ebefdffd79c091fd7c918223aa983f2"}
Jul 22 12:01:44.255368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.255345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:44.255975 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.255945    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bkbtq"
Jul 22 12:01:44.294692 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.294642    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-143.ec2.internal" podStartSLOduration=23.294622222 podStartE2EDuration="23.294622222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:44.273825148 +0000 UTC m=+24.697256200" watchObservedRunningTime="2026-07-22 12:01:44.294622222 +0000 UTC m=+24.718053278"
Jul 22 12:01:44.319265 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.319167    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mlr29" podStartSLOduration=6.048384374 podStartE2EDuration="25.319154077s" totalImagesPullingTime="19.270769703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:44.318929393 +0000 UTC m=+24.742360447" watchObservedRunningTime="2026-07-22 12:01:44.319154077 +0000 UTC m=+24.742585133"
Jul 22 12:01:44.327949 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:44.327922    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 12:01:45.105611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.105454    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T12:01:44.327944438Z","UUID":"082009ad-3265-4140-a94a-9786ea214ed4","Handler":null,"Name":"","Endpoint":""}
Jul 22 12:01:45.107491 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.107456    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 12:01:45.107616 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.107498    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 12:01:45.127910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.127874    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:45.128046 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:45.128016    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:45.258279 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.258242    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69" event={"ID":"22f07980-325d-4c0e-af77-a09fa01d6ec1","Type":"ContainerStarted","Data":"7b5f1df1dbb68c1508e810addbfdec6b86cc8f13dcc708a890c19e6290908dcc"}
Jul 22 12:01:45.261304 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:45.261274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"1bef9dac641d960fb31c2dbcbc2ce411634e609640b98dfc3c77de11779fdb16"}
Jul 22 12:01:46.128364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:46.128331    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:46.128610 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:46.128440    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:46.128841 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:46.128820    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:46.128957 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:46.128928    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:46.265518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:46.265481    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69" event={"ID":"22f07980-325d-4c0e-af77-a09fa01d6ec1","Type":"ContainerStarted","Data":"78069b6ea370f0c6948a09c1ce1c68b8a70443ce08d8f650062e3e6afd42c476"}
Jul 22 12:01:46.288957 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:46.288912    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b2x69" podStartSLOduration=4.275991104 podStartE2EDuration="26.28889767s" totalImagesPullingTime="22.012906566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:46.288440604 +0000 UTC m=+26.711871658" watchObservedRunningTime="2026-07-22 12:01:46.28889767 +0000 UTC m=+26.712328745"
Jul 22 12:01:47.128192 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:47.128118    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:47.128327 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:47.128286    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:47.272516 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:47.272252    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" event={"ID":"d002808c-1a8b-4831-8e37-d35915f20f6e","Type":"ContainerStarted","Data":"244c55f0936653948c73050b5e2c608b81d4a5971ed2400deda4323d0103ecc5"}
Jul 22 12:01:47.320295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:47.320255    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx" podStartSLOduration=8.020501098 podStartE2EDuration="27.320241049s" totalImagesPullingTime="19.299739951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:47.318138271 +0000 UTC m=+27.741569317" watchObservedRunningTime="2026-07-22 12:01:47.320241049 +0000 UTC m=+27.743672104"
Jul 22 12:01:48.128522 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.128489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:48.128688 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.128526    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:48.128688 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:48.128611    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:48.128770 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:48.128719    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:48.276215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.276185    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="a499c0e7108ed33a3cacd8442a321c046ccac7db8fe1c41337d05f5a7b27f6cd" exitCode=0
Jul 22 12:01:48.276996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.276267    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"a499c0e7108ed33a3cacd8442a321c046ccac7db8fe1c41337d05f5a7b27f6cd"}
Jul 22 12:01:48.276996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.276388    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:48.276996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.276453    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:48.276996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.276493    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:48.291814 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.291764    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:48.292043 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:48.292026    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:01:49.128216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.128182    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:49.128433 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:49.128281    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:49.196318 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.196287    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjqtp"]
Jul 22 12:01:49.198900 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.198873    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6ww7v"]
Jul 22 12:01:49.199019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.198976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:49.199111 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:49.199087    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:49.201806 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.201770    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfm2l"]
Jul 22 12:01:49.201902 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.201886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:49.201974 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:49.201959    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:49.279590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:49.279310    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:49.280029 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:49.279895    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:50.282890 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:50.282854    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="e0f5a06f7034f16abb030a9d9c4d31e4cbc371e2e545255340ea639a1ab6c0f5" exitCode=0
Jul 22 12:01:50.283310 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:50.282933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"e0f5a06f7034f16abb030a9d9c4d31e4cbc371e2e545255340ea639a1ab6c0f5"}
Jul 22 12:01:51.128370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:51.128291    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:51.128509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:51.128396    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:51.128509 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:51.128409    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:51.128509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:51.128426    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:51.128676 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:51.128508    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:51.128676 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:51.128613    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:53.128606 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.128574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:53.129017 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.128574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:53.129017 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.128577    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:53.129017 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.128807    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:01:53.129017 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.128667    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6ww7v" podUID="74429ab9-de1f-4270-8848-206b6f733c16"
Jul 22 12:01:53.129017 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.128872    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjqtp" podUID="f865ab54-07aa-4514-9ce8-64f633eec056"
Jul 22 12:01:53.790153 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.790118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:53.790333 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.790196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:53.790333 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.790246    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:53.790333 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.790284    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:53.790333 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.790325    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret podName:f865ab54-07aa-4514-9ce8-64f633eec056 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.790304667 +0000 UTC m=+66.213735714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret") pod "global-pull-secret-syncer-mjqtp" (UID: "f865ab54-07aa-4514-9ce8-64f633eec056") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:01:53.790548 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.790345    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.790334734 +0000 UTC m=+66.213765767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:01:53.890935 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:53.890902    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:53.891088 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.891057    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:01:53.891088 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.891078    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:01:53.891088 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.891088    2565 projected.go:196] Error preparing data for projected volume kube-api-access-w2hwv for pod openshift-network-diagnostics/network-check-target-6ww7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:53.891201 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:53.891131    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv podName:74429ab9-de1f-4270-8848-206b6f733c16 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:25.891119096 +0000 UTC m=+66.314550133 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2hwv" (UniqueName: "kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv") pod "network-check-target-6ww7v" (UID: "74429ab9-de1f-4270-8848-206b6f733c16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:01:54.293464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.293431    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="481b4a0036774a4b543f9180c25e7648e10c6e7294eb54d48a5876812455d07f" exitCode=0
Jul 22 12:01:54.293912 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.293499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"481b4a0036774a4b543f9180c25e7648e10c6e7294eb54d48a5876812455d07f"}
Jul 22 12:01:54.841622 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.841450    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-143.ec2.internal" event="NodeReady"
Jul 22 12:01:54.841823 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.841722    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 12:01:54.886005 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.885965    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:01:54.906905 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.906873    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"]
Jul 22 12:01:54.907072 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.906976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.909546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.909517    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 12:01:54.909653 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.909587    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 12:01:54.909653 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.909596    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 12:01:54.909744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.909585    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t9ltr\""
Jul 22 12:01:54.915220 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.915196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 12:01:54.929063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.929030    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"]
Jul 22 12:01:54.929176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.929160    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:54.931163 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.931139    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 12:01:54.931276 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.931147    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 12:01:54.931444 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.931426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 22 12:01:54.931551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.931495    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 12:01:54.949106 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.949085    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"]
Jul 22 12:01:54.949221 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.949205    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:54.951469 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.951448    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 22 12:01:54.951557 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.951486    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-twlgb\""
Jul 22 12:01:54.963202 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.963182    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vmvx2"]
Jul 22 12:01:54.963327 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.963312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:54.965437 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.965417    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 22 12:01:54.965867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.965845    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 22 12:01:54.965867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.965861    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 22 12:01:54.965990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.965848    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 22 12:01:54.981191 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981172    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"]
Jul 22 12:01:54.981285 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981200    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"]
Jul 22 12:01:54.981285 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981211    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:01:54.981285 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981224    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"]
Jul 22 12:01:54.981285 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981238    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7lb7c"]
Jul 22 12:01:54.981419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.981343    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:54.983851 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.983825    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 12:01:54.983932 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.983899    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 12:01:54.983932 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.983917    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q9wgb\""
Jul 22 12:01:54.995237 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.995220    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmvx2"]
Jul 22 12:01:54.995324 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.995251    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lb7c"]
Jul 22 12:01:54.995386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.995344    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:54.997898 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.997879    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 12:01:54.998301 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998281    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998462    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 12:01:54.998567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:54.998759 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998532    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-82grb\""
Jul 22 12:01:54.998759 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998534    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 12:01:54.998994 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:54.998960    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmfbk\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099694 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099694 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.099694 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8431cc2e-fbfe-43e2-a55f-5e223e92435e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099833    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmfbk\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.099982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099918    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddd8680-6267-4a72-b3e5-0017fa85f47d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.099984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkpvq\" (UniqueName: \"kubernetes.io/projected/4f461d01-9d3e-4ba4-9dd7-8207c974e321-kube-api-access-nkpvq\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100033    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvl4\" (UniqueName: \"kubernetes.io/projected/1ddd8680-6267-4a72-b3e5-0017fa85f47d-kube-api-access-wxvl4\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6baf9ac9-5cca-4dc4-8844-c658420892c8-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2pbj\" (UniqueName: \"kubernetes.io/projected/6baf9ac9-5cca-4dc4-8844-c658420892c8-kube-api-access-s2pbj\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.100208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b7kr\" (UniqueName: \"kubernetes.io/projected/8431cc2e-fbfe-43e2-a55f-5e223e92435e-kube-api-access-2b7kr\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100302    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztsw\" (UniqueName: \"kubernetes.io/projected/81ed3d62-5302-4d22-aad4-b2aadc783b87-kube-api-access-bztsw\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.100414    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6baf9ac9-5cca-4dc4-8844-c658420892c8-tmp\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.100429    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f461d01-9d3e-4ba4-9dd7-8207c974e321-config-volume\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.100554 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.100497    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:01:55.600477889 +0000 UTC m=+36.023908937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:01:55.101007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f461d01-9d3e-4ba4-9dd7-8207c974e321-tmp-dir\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.101007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81ed3d62-5302-4d22-aad4-b2aadc783b87-cert\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.101007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100705    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.101007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.100735    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.103713 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.103683    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:01:55.104234 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.104213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.104316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.104241    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.111777 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.111748    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.111909 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.111861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmfbk\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.127857 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.127832    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:01:55.127968 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.127832    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:01:55.128029 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.127832    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:01:55.130396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130380    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:01:55.130486 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130467    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:01:55.130581 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130542    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-856ws\""
Jul 22 12:01:55.130628 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130606    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:01:55.130822 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130808    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:01:55.130822 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.130813    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4n945\""
Jul 22 12:01:55.201330 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b7kr\" (UniqueName: \"kubernetes.io/projected/8431cc2e-fbfe-43e2-a55f-5e223e92435e-kube-api-access-2b7kr\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201388    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztsw\" (UniqueName: \"kubernetes.io/projected/81ed3d62-5302-4d22-aad4-b2aadc783b87-kube-api-access-bztsw\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6baf9ac9-5cca-4dc4-8844-c658420892c8-tmp\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f461d01-9d3e-4ba4-9dd7-8207c974e321-config-volume\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.201466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f461d01-9d3e-4ba4-9dd7-8207c974e321-tmp-dir\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.201745 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81ed3d62-5302-4d22-aad4-b2aadc783b87-cert\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.201745 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.201745 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.201534    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:01:55.201745 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.201601    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:55.701579366 +0000 UTC m=+36.125010399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:01:55.201745 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8431cc2e-fbfe-43e2-a55f-5e223e92435e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddd8680-6267-4a72-b3e5-0017fa85f47d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkpvq\" (UniqueName: \"kubernetes.io/projected/4f461d01-9d3e-4ba4-9dd7-8207c974e321-kube-api-access-nkpvq\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201853    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvl4\" (UniqueName: \"kubernetes.io/projected/1ddd8680-6267-4a72-b3e5-0017fa85f47d-kube-api-access-wxvl4\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6baf9ac9-5cca-4dc4-8844-c658420892c8-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6baf9ac9-5cca-4dc4-8844-c658420892c8-tmp\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.202019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201927    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f461d01-9d3e-4ba4-9dd7-8207c974e321-tmp-dir\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.202388 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.201930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2pbj\" (UniqueName: \"kubernetes.io/projected/6baf9ac9-5cca-4dc4-8844-c658420892c8-kube-api-access-s2pbj\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.202388 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.202305    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4f461d01-9d3e-4ba4-9dd7-8207c974e321-config-volume\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.203148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.203123    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddd8680-6267-4a72-b3e5-0017fa85f47d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.204528 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.204498    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.205035 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.204899    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.205312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.205283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-ca\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.205553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.205527    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1ddd8680-6267-4a72-b3e5-0017fa85f47d-hub\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.205654 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.205626    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8431cc2e-fbfe-43e2-a55f-5e223e92435e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.206636 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.206596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81ed3d62-5302-4d22-aad4-b2aadc783b87-cert\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.207052 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.207033    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6baf9ac9-5cca-4dc4-8844-c658420892c8-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.215487 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.215459    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztsw\" (UniqueName: \"kubernetes.io/projected/81ed3d62-5302-4d22-aad4-b2aadc783b87-kube-api-access-bztsw\") pod \"ingress-canary-7lb7c\" (UID: \"81ed3d62-5302-4d22-aad4-b2aadc783b87\") " pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.215738 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.215716    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b7kr\" (UniqueName: \"kubernetes.io/projected/8431cc2e-fbfe-43e2-a55f-5e223e92435e-kube-api-access-2b7kr\") pod \"managed-serviceaccount-addon-agent-58f4995884-lnh8l\" (UID: \"8431cc2e-fbfe-43e2-a55f-5e223e92435e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.216428 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.216402    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvl4\" (UniqueName: \"kubernetes.io/projected/1ddd8680-6267-4a72-b3e5-0017fa85f47d-kube-api-access-wxvl4\") pod \"cluster-proxy-proxy-agent-85796447-hb9tp\" (UID: \"1ddd8680-6267-4a72-b3e5-0017fa85f47d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.216714 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.216592    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2pbj\" (UniqueName: \"kubernetes.io/projected/6baf9ac9-5cca-4dc4-8844-c658420892c8-kube-api-access-s2pbj\") pod \"klusterlet-addon-workmgr-6bc7c7cf75-nrn92\" (UID: \"6baf9ac9-5cca-4dc4-8844-c658420892c8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.217303 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.217281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkpvq\" (UniqueName: \"kubernetes.io/projected/4f461d01-9d3e-4ba4-9dd7-8207c974e321-kube-api-access-nkpvq\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.217923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.217880    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:01:55.238888 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.238861    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:01:55.267504 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.267477    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"
Jul 22 12:01:55.273684 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.273660    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:01:55.310167 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.310139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lb7c"
Jul 22 12:01:55.494419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.494388    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"]
Jul 22 12:01:55.496761 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:55.496723    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6baf9ac9_5cca_4dc4_8844_c658420892c8.slice/crio-c7f7bb1b7abd908f446346e1954183fd1dac28e3f800b81ae305cce5460856e0 WatchSource:0}: Error finding container c7f7bb1b7abd908f446346e1954183fd1dac28e3f800b81ae305cce5460856e0: Status 404 returned error can't find the container with id c7f7bb1b7abd908f446346e1954183fd1dac28e3f800b81ae305cce5460856e0
Jul 22 12:01:55.497592 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.496977    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l"]
Jul 22 12:01:55.504036 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:55.504007    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8431cc2e_fbfe_43e2_a55f_5e223e92435e.slice/crio-cd8104ceb44480e7297ce2a5659ddb1ee8385d17e95c0ba28115f6a2e6169101 WatchSource:0}: Error finding container cd8104ceb44480e7297ce2a5659ddb1ee8385d17e95c0ba28115f6a2e6169101: Status 404 returned error can't find the container with id cd8104ceb44480e7297ce2a5659ddb1ee8385d17e95c0ba28115f6a2e6169101
Jul 22 12:01:55.517295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.517269    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"]
Jul 22 12:01:55.520779 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:55.520753    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ddd8680_6267_4a72_b3e5_0017fa85f47d.slice/crio-0ee69bf35bcbf152b3508b3ce417b29add15adcedfa70893736fe8f326cc5424 WatchSource:0}: Error finding container 0ee69bf35bcbf152b3508b3ce417b29add15adcedfa70893736fe8f326cc5424: Status 404 returned error can't find the container with id 0ee69bf35bcbf152b3508b3ce417b29add15adcedfa70893736fe8f326cc5424
Jul 22 12:01:55.550305 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.550273    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lb7c"]
Jul 22 12:01:55.553260 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:01:55.553235    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ed3d62_5302_4d22_aad4_b2aadc783b87.slice/crio-ede0cb03baaf8fd2ecfcfeaff929e11de448b131448e2855a59f6dc6e0dea5fd WatchSource:0}: Error finding container ede0cb03baaf8fd2ecfcfeaff929e11de448b131448e2855a59f6dc6e0dea5fd: Status 404 returned error can't find the container with id ede0cb03baaf8fd2ecfcfeaff929e11de448b131448e2855a59f6dc6e0dea5fd
Jul 22 12:01:55.608507 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.608434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:55.608660 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.608588    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:01:55.608660 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.608616    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:01:55.608745 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.608686    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:01:56.60866617 +0000 UTC m=+37.032097219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:01:55.709355 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:55.709320    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:55.709523 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.709506    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:01:55.709609 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:55.709596    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:56.709574144 +0000 UTC m=+37.133005194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:01:56.303126 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.303018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lb7c" event={"ID":"81ed3d62-5302-4d22-aad4-b2aadc783b87","Type":"ContainerStarted","Data":"ede0cb03baaf8fd2ecfcfeaff929e11de448b131448e2855a59f6dc6e0dea5fd"}
Jul 22 12:01:56.304550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.304496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l" event={"ID":"8431cc2e-fbfe-43e2-a55f-5e223e92435e","Type":"ContainerStarted","Data":"cd8104ceb44480e7297ce2a5659ddb1ee8385d17e95c0ba28115f6a2e6169101"}
Jul 22 12:01:56.306313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.306255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92" event={"ID":"6baf9ac9-5cca-4dc4-8844-c658420892c8","Type":"ContainerStarted","Data":"c7f7bb1b7abd908f446346e1954183fd1dac28e3f800b81ae305cce5460856e0"}
Jul 22 12:01:56.307880 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.307847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerStarted","Data":"0ee69bf35bcbf152b3508b3ce417b29add15adcedfa70893736fe8f326cc5424"}
Jul 22 12:01:56.616881 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.616782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:56.617533 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:56.617013    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:01:56.617533 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:56.617030    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:01:56.617533 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:56.617088    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:01:58.617068293 +0000 UTC m=+39.040499330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:01:56.718203 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:56.717611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:56.718203 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:56.718070    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:01:56.718203 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:56.718136    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:01:58.718117508 +0000 UTC m=+39.141548556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:01:58.638174 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:58.638119    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:01:58.638690 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:58.638346    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:01:58.638690 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:58.638362    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:01:58.638690 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:58.638425    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:02:02.638403303 +0000 UTC m=+43.061834359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:01:58.739135 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:58.739090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:01:58.739291 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:58.739266    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:01:58.739368 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:01:58.739343    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:02.739322692 +0000 UTC m=+43.162753738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:01:58.944893 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:58.944629    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:01:59.046128 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:59.046052    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:01:59.144469 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:59.144403    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:01:59.926007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:01:59.925971    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:00.048704 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:00.048666    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:00.734019 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:00.733975    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:01.442163 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:01.442128    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:02.669060 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:02.669021    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:02:02.669514 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:02.669178    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:02:02.669514 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:02.669201    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:02:02.669514 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:02.669293    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:02:10.669273357 +0000 UTC m=+51.092704394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:02:02.770014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:02.769975    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:02:02.770198 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:02.770145    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:02.770248 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:02.770215    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:10.770195589 +0000 UTC m=+51.193626626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:05.339602 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.339483    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l" event={"ID":"8431cc2e-fbfe-43e2-a55f-5e223e92435e","Type":"ContainerStarted","Data":"839057b6a93dd89bd4267c66709ea0abcd9cce01f775293a701bc577a757729a"}
Jul 22 12:02:05.343379 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.343323    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92" event={"ID":"6baf9ac9-5cca-4dc4-8844-c658420892c8","Type":"ContainerStarted","Data":"62a086989bc724c7ad454b82230fc23e4af8df1737f99fa3d53bbfb132be3456"}
Jul 22 12:02:05.343809 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.343720    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:02:05.344973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.344936    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92" podUID="6baf9ac9-5cca-4dc4-8844-c658420892c8" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.7:8000/readyz\": dial tcp 10.132.0.7:8000: connect: connection refused"
Jul 22 12:02:05.377985 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.377924    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58f4995884-lnh8l" podStartSLOduration=23.696783751 podStartE2EDuration="33.37790376s" totalImagesPullingTime="9.681120009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:05.359824382 +0000 UTC m=+45.783255437" watchObservedRunningTime="2026-07-22 12:02:05.37790376 +0000 UTC m=+45.801334818"
Jul 22 12:02:05.378175 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:05.378146    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92" podStartSLOduration=23.692019853 podStartE2EDuration="33.37813806s" totalImagesPullingTime="9.686118207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:05.377149484 +0000 UTC m=+45.800580540" watchObservedRunningTime="2026-07-22 12:02:05.37813806 +0000 UTC m=+45.801569106"
Jul 22 12:02:06.347408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.347369    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lb7c" event={"ID":"81ed3d62-5302-4d22-aad4-b2aadc783b87","Type":"ContainerStarted","Data":"8d41b20f7bbbaa004524bda29ff8abe0fbc2d7aeeedbc6be9222b349b5cc92d2"}
Jul 22 12:02:06.349887 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.349863    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="8d92668f3992bdf4b3d1a246e9db96fab92c5e0748bd0f6fbb5300fd82b69d3d" exitCode=0
Jul 22 12:02:06.350001 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.349935    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"8d92668f3992bdf4b3d1a246e9db96fab92c5e0748bd0f6fbb5300fd82b69d3d"}
Jul 22 12:02:06.351992 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.351967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerStarted","Data":"94d76f7af7c49f93ea9ce6f8b57156ffdb67e19bf31abf3d5b7c493b1d9aa0cf"}
Jul 22 12:02:06.352093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.352006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerStarted","Data":"e5a35187d5e06c60fda110738e24808d6dd63a001850a2e4ce549be72cefba31"}
Jul 22 12:02:06.352093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.352026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerStarted","Data":"351ea08c9296f2026ffe2edd9f882d92d27095c5e822393ca435651586cfb941"}
Jul 22 12:02:06.353120 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.353101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc7c7cf75-nrn92"
Jul 22 12:02:06.365739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.365693    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7lb7c" podStartSLOduration=2.732803146 podStartE2EDuration="12.365676229s" totalImagesPullingTime="9.632873083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:06.365093511 +0000 UTC m=+46.788524565" watchObservedRunningTime="2026-07-22 12:02:06.365676229 +0000 UTC m=+46.789107285"
Jul 22 12:02:06.441080 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:06.441022    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" podStartSLOduration=24.760316193 podStartE2EDuration="34.441001602s" totalImagesPullingTime="9.680685409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:06.439726266 +0000 UTC m=+46.863157322" watchObservedRunningTime="2026-07-22 12:02:06.441001602 +0000 UTC m=+46.864432657"
Jul 22 12:02:07.357609 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:07.357573    2565 generic.go:356] "Generic (PLEG): container finished" podID="2481599b-ebae-4b9a-b7f6-90f59129531a" containerID="466167504f65fc9d362033da233648507fefa15fcd4e609d97fc0ad94d8ef0c7" exitCode=0
Jul 22 12:02:07.358012 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:07.357676    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerDied","Data":"466167504f65fc9d362033da233648507fefa15fcd4e609d97fc0ad94d8ef0c7"}
Jul 22 12:02:08.365267 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:08.365230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" event={"ID":"2481599b-ebae-4b9a-b7f6-90f59129531a","Type":"ContainerStarted","Data":"2872440c8e1a7e22294be20d9e38041508d86ce5ae413c0d58801409d32a5977"}
Jul 22 12:02:08.395381 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:08.395326    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rwrfg" podStartSLOduration=10.787664752 podStartE2EDuration="49.395311406s" totalImagesPullingTime="38.607646654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:08.394306892 +0000 UTC m=+48.817737942" watchObservedRunningTime="2026-07-22 12:02:08.395311406 +0000 UTC m=+48.818742460"
Jul 22 12:02:08.542533 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:08.542496    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:08.640071 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:08.639987    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:10.733076 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:10.733035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:02:10.733461 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:10.733150    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:02:10.733461 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:10.733161    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:02:10.733461 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:10.733208    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:02:26.733195499 +0000 UTC m=+67.156626532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:02:10.833332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:10.833297    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:02:10.833468 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:10.833444    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:10.833522 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:10.833512    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:26.833497358 +0000 UTC m=+67.256928397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:11.527628 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:11.527591    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:11.629133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:11.629093    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:12.221532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:12.221496    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:12.321925 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:12.321882    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:13.801509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:13.801464    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:13.895375 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:13.895336    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:14.048267 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:14.048234    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:16.130673 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:16.130636    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:16.227258 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:16.227220    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:16.723153 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:16.723123    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:20.293216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:20.293190    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcthx"
Jul 22 12:02:20.453181 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:20.453146    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:20.530324 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:20.530292    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:25.842769 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.842731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:02:25.843217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.842826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:02:25.845342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.845324    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:02:25.845398 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.845326    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:02:25.853478 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:25.853454    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 12:02:25.853604 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:25.853522    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:29.85349964 +0000 UTC m=+130.276930672 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : secret "metrics-daemon-secret" not found
Jul 22 12:02:25.856533 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.856510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f865ab54-07aa-4514-9ce8-64f633eec056-original-pull-secret\") pod \"global-pull-secret-syncer-mjqtp\" (UID: \"f865ab54-07aa-4514-9ce8-64f633eec056\") " pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:02:25.943123 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.943093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:02:25.945775 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.945760    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:02:25.956362 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.956341    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:02:25.966539 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:25.966510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2hwv\" (UniqueName: \"kubernetes.io/projected/74429ab9-de1f-4270-8848-206b6f733c16-kube-api-access-w2hwv\") pod \"network-check-target-6ww7v\" (UID: \"74429ab9-de1f-4270-8848-206b6f733c16\") " pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:02:26.045472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.045440    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-856ws\""
Jul 22 12:02:26.048193 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.048178    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjqtp"
Jul 22 12:02:26.053206 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.053188    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:02:26.185724 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.185692    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6ww7v"]
Jul 22 12:02:26.189024 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:02:26.188998    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74429ab9_de1f_4270_8848_206b6f733c16.slice/crio-43a211d06ab2774d0773a4c2ecddb9503412ce5664d0c46e0bfa15dee71fff0d WatchSource:0}: Error finding container 43a211d06ab2774d0773a4c2ecddb9503412ce5664d0c46e0bfa15dee71fff0d: Status 404 returned error can't find the container with id 43a211d06ab2774d0773a4c2ecddb9503412ce5664d0c46e0bfa15dee71fff0d
Jul 22 12:02:26.201913 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.201885    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjqtp"]
Jul 22 12:02:26.205299 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:02:26.205277    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf865ab54_07aa_4514_9ce8_64f633eec056.slice/crio-7455e41d743b5c375191ca6b5e87d92d11fec13b195e63e693b4316eba5c09e7 WatchSource:0}: Error finding container 7455e41d743b5c375191ca6b5e87d92d11fec13b195e63e693b4316eba5c09e7: Status 404 returned error can't find the container with id 7455e41d743b5c375191ca6b5e87d92d11fec13b195e63e693b4316eba5c09e7
Jul 22 12:02:26.376614 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.376481    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:26.419514 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.419481    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6ww7v" event={"ID":"74429ab9-de1f-4270-8848-206b6f733c16","Type":"ContainerStarted","Data":"43a211d06ab2774d0773a4c2ecddb9503412ce5664d0c46e0bfa15dee71fff0d"}
Jul 22 12:02:26.420557 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.420526    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjqtp" event={"ID":"f865ab54-07aa-4514-9ce8-64f633eec056","Type":"ContainerStarted","Data":"7455e41d743b5c375191ca6b5e87d92d11fec13b195e63e693b4316eba5c09e7"}
Jul 22 12:02:26.749943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.749853    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:02:26.750103 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:26.750024    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:02:26.750103 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:26.750048    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:02:26.750216 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:26.750116    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:02:58.750094108 +0000 UTC m=+99.173525157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:02:26.850301 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:26.850259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:02:26.850721 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:26.850412    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:26.850721 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:26.850473    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:02:58.850456904 +0000 UTC m=+99.273887949 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:02:30.552842 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:30.552815    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:30.889734 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:30.889628    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:30.980438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:30.980396    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:31.440438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:31.440401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjqtp" event={"ID":"f865ab54-07aa-4514-9ce8-64f633eec056","Type":"ContainerStarted","Data":"e9902a7eb2b4131bb071b6a099757fdee96c30915063a295923b3bebe2326f41"}
Jul 22 12:02:31.441814 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:31.441775    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6ww7v" event={"ID":"74429ab9-de1f-4270-8848-206b6f733c16","Type":"ContainerStarted","Data":"b91cb3f5794c63e3e8ae15eb19f61a9e50e8666c8b0736a565f468174772b390"}
Jul 22 12:02:31.441942 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:31.441921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:02:31.462609 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:31.462564    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mjqtp" podStartSLOduration=67.134891158 podStartE2EDuration="1m11.462549267s" totalImagesPullingTime="4.327658109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:31.461196594 +0000 UTC m=+71.884627649" watchObservedRunningTime="2026-07-22 12:02:31.462549267 +0000 UTC m=+71.885980322"
Jul 22 12:02:31.479981 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:31.479926    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6ww7v" podStartSLOduration=68.142166677 podStartE2EDuration="1m12.47991149s" totalImagesPullingTime="4.337744813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:02:31.479694165 +0000 UTC m=+71.903125221" watchObservedRunningTime="2026-07-22 12:02:31.47991149 +0000 UTC m=+71.903342544"
Jul 22 12:02:32.212885 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:32.212846    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:36.900074 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:36.900034    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:36.977113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:36.977069    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:49.318544 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:49.318392    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:52.831116 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:52.831083    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:52.932642 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:52.932598    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:02:58.775490 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:58.775455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:02:58.775845 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:58.775579    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:02:58.775845 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:58.775591    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f885957f8-tc8t5: secret "image-registry-tls" not found
Jul 22 12:02:58.775845 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:58.775657    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls podName:da40d227-684c-48f5-97a5-0ac229ee4a8a nodeName:}" failed. No retries permitted until 2026-07-22 12:04:02.775643064 +0000 UTC m=+163.199074096 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls") pod "image-registry-7f885957f8-tc8t5" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a") : secret "image-registry-tls" not found
Jul 22 12:02:58.876360 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:02:58.876326    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:02:58.876466 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:58.876453    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:02:58.876513 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:02:58.876504    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls podName:4f461d01-9d3e-4ba4-9dd7-8207c974e321 nodeName:}" failed. No retries permitted until 2026-07-22 12:04:02.876491425 +0000 UTC m=+163.299922463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls") pod "dns-default-vmvx2" (UID: "4f461d01-9d3e-4ba4-9dd7-8207c974e321") : secret "dns-default-metrics-tls" not found
Jul 22 12:03:02.446688 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:02.446655    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6ww7v"
Jul 22 12:03:02.759113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:02.759018    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:02.857489 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:02.857448    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:06.785458 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.785416    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-64xp7"]
Jul 22 12:03:06.788389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.788365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:06.805994 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.805974    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 12:03:06.806102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.806000    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 12:03:06.806102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.806042    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 12:03:06.806102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.806095    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 12:03:06.815474 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.815444    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-64xp7"]
Jul 22 12:03:06.819332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.819301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmstd\""
Jul 22 12:03:06.854913 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.854886    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:06.935596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.935563    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:06.940581 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.940555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-crio-socket\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:06.940698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.940587    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:06.940698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.940626    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:06.940698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.940655    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lxnl\" (UniqueName: \"kubernetes.io/projected/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-api-access-5lxnl\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:06.940698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:06.940680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-data-volume\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041337 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041256    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-crio-socket\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041337 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041337 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041378    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-crio-socket\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041503 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.041383    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.041503 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.041460    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:07.541443598 +0000 UTC m=+107.964874634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.041503 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.041401    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:07.041503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lxnl\" (UniqueName: \"kubernetes.io/projected/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-api-access-5lxnl\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041658 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.041523    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:07.541511422 +0000 UTC m=+107.964942455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:07.041658 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-data-volume\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.041899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.041886    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-data-volume\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.050330 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.050311    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lxnl\" (UniqueName: \"kubernetes.io/projected/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-api-access-5lxnl\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.544573 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.544534    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.544730 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:07.544586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:07.544730 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.544665    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.544730 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.544718    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:07.544852 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.544744    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:08.544727806 +0000 UTC m=+108.968158843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:07.544852 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:07.544769    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:08.544757047 +0000 UTC m=+108.968188080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.553559 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:08.553524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:08.553976 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:08.553568    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:08.553976 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:08.553645    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:08.553976 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:08.553682    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.553976 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:08.553710    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.553692319 +0000 UTC m=+110.977123368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:08.553976 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:08.553725    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:10.553718285 +0000 UTC m=+110.977149321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:08.963820 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:08.963730    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmpph_8edb1ee9-df00-4ad9-b693-af21cdfac67e/dns-node-resolver/0.log"
Jul 22 12:03:09.364302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:09.364277    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-58ss7_807e7e23-00a5-4e02-addf-9264b282db03/node-ca/0.log"
Jul 22 12:03:10.164515 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:10.164484    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7lb7c_81ed3d62-5302-4d22-aad4-b2aadc783b87/serve-healthcheck-canary/0.log"
Jul 22 12:03:10.566963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:10.566924    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:10.567209 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:10.566981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:10.567209 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:10.567046    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:10.567209 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:10.567081    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:10.567209 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:10.567104    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:14.567088367 +0000 UTC m=+114.990519400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:10.567209 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:10.567131    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:14.56711855 +0000 UTC m=+114.990549587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:12.015278 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:12.015238    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:12.080821 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:12.080753    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:14.593409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:14.593376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:14.593778 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:14.593419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:14.593778 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:14.593487    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:03:14.593778 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:14.593522    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:14.593778 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:14.593543    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:22.59352881 +0000 UTC m=+123.016959843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : configmap "kube-rbac-proxy" not found
Jul 22 12:03:14.593778 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:14.593564    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:22.593553296 +0000 UTC m=+123.016984329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:16.820442 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:16.820405    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:16.890005 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:16.889971    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:17.302291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:17.302257    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:17.388555 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:17.388518    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:17.456893 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:17.456862    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:17.534504 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:17.534474    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:17.636151 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:17.636063    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:18.358165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:18.358129    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:19.190809 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:19.190753    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:19.268320 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:19.268287    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:20.076627 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:20.076596    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:21.163130 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:21.163098    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:22.464613 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.464578    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:22.576178 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.576144    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:22.652676 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.652641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:22.652863 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.652688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:22.652863 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:22.652849    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:22.652940 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:22.652915    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls podName:af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc nodeName:}" failed. No retries permitted until 2026-07-22 12:03:38.652901215 +0000 UTC m=+139.076332248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-64xp7" (UID: "af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:03:22.653165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.653146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:22.674744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.674709    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:22.768226 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.768196    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:22.877970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:22.877943    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:23.353312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:23.353272    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:23.741430 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:23.741318    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:25.402948 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:25.402884    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:25.503608 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:25.503562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:25.737304 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:25.737222    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.063978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.063910    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.165897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.165860    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.330815 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.330710    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.430999 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.430964    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.530553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.530522    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:26.736435 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:26.736345    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:27.223335 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:27.223305    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:27.325041 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:27.324992    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:27.591267 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:27.591233    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:28.133774 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:28.133735    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:28.328745 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:28.328710    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:28.742462 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:28.742427    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:28.854317 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:28.854287    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:28.955112 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:28.955080    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:29.532600 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:29.532563    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:29.912755 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:29.912668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:03:29.913140 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:29.912835    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 12:03:29.913140 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:29.912910    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs podName:32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773 nodeName:}" failed. No retries permitted until 2026-07-22 12:05:31.912895014 +0000 UTC m=+252.336326048 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs") pod "network-metrics-daemon-kfm2l" (UID: "32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773") : secret "metrics-daemon-secret" not found
Jul 22 12:03:29.943213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:29.943175    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:30.180516 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:30.180431    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:30.739996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:30.739960    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:31.344086 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:31.344056    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:31.538802 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:31.538767    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:31.932743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:31.932711    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:32.462503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:32.462470    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:32.994424 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:32.994389    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:33.072173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:33.072141    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:33.147921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:33.147887    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:35.275686 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:35.275647    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" podUID="1ddd8680-6267-4a72-b3e5-0017fa85f47d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:03:36.206836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.206776    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:36.306082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.306047    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:36.387601 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.387559    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:36.522441 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.522413    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:36.678742 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.678711    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:36.750662 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:36.750633    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:37.111839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:37.111807    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:37.212700 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:37.212664    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:37.312918 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:37.312888    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:37.698924 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:37.698894    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.022296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.022267    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.094988 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.094957    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.195264 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.195232    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.295495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.295419    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.543906 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.543866    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.644348 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.644263    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:38.680273 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.680246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:38.682494 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.682475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-64xp7\" (UID: \"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc\") " pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:38.898764 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.898684    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmstd\""
Jul 22 12:03:38.906329 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:38.906285    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-64xp7"
Jul 22 12:03:39.032528 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:39.032505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-64xp7"]
Jul 22 12:03:39.035844 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:03:39.035810    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf99f0d9_bdde_4e7e_93f9_0db1c4e3bcdc.slice/crio-9e2eb7a19e9676610bde14e2ac68a11fe8ce10ba42dc5956ca3910e7d1dc97f9 WatchSource:0}: Error finding container 9e2eb7a19e9676610bde14e2ac68a11fe8ce10ba42dc5956ca3910e7d1dc97f9: Status 404 returned error can't find the container with id 9e2eb7a19e9676610bde14e2ac68a11fe8ce10ba42dc5956ca3910e7d1dc97f9
Jul 22 12:03:39.648135 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:39.648101    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-64xp7" event={"ID":"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc","Type":"ContainerStarted","Data":"9e2eb7a19e9676610bde14e2ac68a11fe8ce10ba42dc5956ca3910e7d1dc97f9"}
Jul 22 12:03:40.514396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:40.514353    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:40.592589 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:40.592544    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:42.267266 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:42.267177    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:43.129603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:43.129565    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:44.079594 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:44.079561    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:44.205537 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:44.205501    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:44.664937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:44.664907    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-64xp7" event={"ID":"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc","Type":"ContainerStarted","Data":"78bd3e2639b0da81a4f68833c2fe0d458a9d4dcf357893e4e415acb6b10fc220"}
Jul 22 12:03:45.275360 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:45.275322    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" podUID="1ddd8680-6267-4a72-b3e5-0017fa85f47d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:03:46.341211 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:46.341076    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:46.463328 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:46.463292    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:47.336343 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:47.336303    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:48.078642 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:48.078596    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:48.679683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:48.679649    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-64xp7" event={"ID":"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc","Type":"ContainerStarted","Data":"ceca6c5e43d4f5ea6b35ba106c29a5b0b73b8e1729c389e6f9f06f7621f41780"}
Jul 22 12:03:48.874810 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:48.874711    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:49.699253 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:49.699218    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:49.798696 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:49.798662    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:50.206478 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:50.206440    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:50.307384 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:50.307351    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:51.456492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:51.456449    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:52.697419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:52.697379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-64xp7" event={"ID":"af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc","Type":"ContainerStarted","Data":"b5f673f6ee1814070fa5e34bf789564ed3b430d33cf0177065f97dd430d44b88"}
Jul 22 12:03:52.716264 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:52.716210    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-64xp7" podStartSLOduration=33.93582678 podStartE2EDuration="46.716194252s" totalImagesPullingTime="12.780367472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:03:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:52.714828365 +0000 UTC m=+153.138259420" watchObservedRunningTime="2026-07-22 12:03:52.716194252 +0000 UTC m=+153.139625308"
Jul 22 12:03:53.106430 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.106396    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:53.207710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.207675    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:53.867216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.867182    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:53.872342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.872316    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2wh82"]
Jul 22 12:03:53.875584 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.875568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.878408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.878382    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 12:03:53.880007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.879981    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fbpwm\""
Jul 22 12:03:53.880128 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.880073    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 12:03:53.880128 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.880084    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 12:03:53.880128 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.880118    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 12:03:53.880298 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.880086    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 12:03:53.880298 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.880179    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 12:03:53.898042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-textfile\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898064    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-wtmp\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-sys\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-metrics-client-ca\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt8t9\" (UniqueName: \"kubernetes.io/projected/bba7f867-311c-41a7-a02a-22badc2d7c38-kube-api-access-vt8t9\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898205    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-root\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.898357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.898242    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.954741 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.954706    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:53.998993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.998967    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-textfile\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-wtmp\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-sys\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999087    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-metrics-client-ca\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt8t9\" (UniqueName: \"kubernetes.io/projected/bba7f867-311c-41a7-a02a-22badc2d7c38-kube-api-access-vt8t9\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-root\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999169    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-sys\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:53.999129    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999205    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-root\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-wtmp\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999403 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:53.999256    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls podName:bba7f867-311c-41a7-a02a-22badc2d7c38 nodeName:}" failed. No retries permitted until 2026-07-22 12:03:54.49923504 +0000 UTC m=+154.922666097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls") pod "node-exporter-2wh82" (UID: "bba7f867-311c-41a7-a02a-22badc2d7c38") : secret "node-exporter-tls" not found
Jul 22 12:03:53.999759 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-textfile\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999759 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:53.999877 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:53.999776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bba7f867-311c-41a7-a02a-22badc2d7c38-metrics-client-ca\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.001431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.001411    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.010165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.010144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt8t9\" (UniqueName: \"kubernetes.io/projected/bba7f867-311c-41a7-a02a-22badc2d7c38-kube-api-access-vt8t9\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.145333 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.145255    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:54.245982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.245950    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:54.345397 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.345353    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:54.504245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.504157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.506389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.506357    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bba7f867-311c-41a7-a02a-22badc2d7c38-node-exporter-tls\") pod \"node-exporter-2wh82\" (UID: \"bba7f867-311c-41a7-a02a-22badc2d7c38\") " pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.751600 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.751562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:54.784193 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.784164    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2wh82"
Jul 22 12:03:54.794967 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:03:54.794927    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba7f867_311c_41a7_a02a_22badc2d7c38.slice/crio-0d936138bc8362a1e4c0b22f5e50e06e4e9318200904e27c3446a788df72f554 WatchSource:0}: Error finding container 0d936138bc8362a1e4c0b22f5e50e06e4e9318200904e27c3446a788df72f554: Status 404 returned error can't find the container with id 0d936138bc8362a1e4c0b22f5e50e06e4e9318200904e27c3446a788df72f554
Jul 22 12:03:54.864452 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.864422    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:54.965676 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:54.965646    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:55.189252 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.189156    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:55.274837 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.274778    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" podUID="1ddd8680-6267-4a72-b3e5-0017fa85f47d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:03:55.275011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.274880    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp"
Jul 22 12:03:55.275405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.275369    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"94d76f7af7c49f93ea9ce6f8b57156ffdb67e19bf31abf3d5b7c493b1d9aa0cf"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 22 12:03:55.275511 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.275427    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" podUID="1ddd8680-6267-4a72-b3e5-0017fa85f47d" containerName="service-proxy" containerID="cri-o://94d76f7af7c49f93ea9ce6f8b57156ffdb67e19bf31abf3d5b7c493b1d9aa0cf" gracePeriod=30
Jul 22 12:03:55.391125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.391093    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:55.541846 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.541816    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:55.707928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.707897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2wh82" event={"ID":"bba7f867-311c-41a7-a02a-22badc2d7c38","Type":"ContainerStarted","Data":"0d936138bc8362a1e4c0b22f5e50e06e4e9318200904e27c3446a788df72f554"}
Jul 22 12:03:55.710852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.710751    2565 generic.go:356] "Generic (PLEG): container finished" podID="1ddd8680-6267-4a72-b3e5-0017fa85f47d" containerID="94d76f7af7c49f93ea9ce6f8b57156ffdb67e19bf31abf3d5b7c493b1d9aa0cf" exitCode=2
Jul 22 12:03:55.710852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.710819    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerDied","Data":"94d76f7af7c49f93ea9ce6f8b57156ffdb67e19bf31abf3d5b7c493b1d9aa0cf"}
Jul 22 12:03:55.711157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.710871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85796447-hb9tp" event={"ID":"1ddd8680-6267-4a72-b3e5-0017fa85f47d","Type":"ContainerStarted","Data":"2b15cb26a38e13339d695f416dd7d5c8896ae97e833591808c7e45a2e2aef51f"}
Jul 22 12:03:55.741038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:55.741002    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:56.715640 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:56.715601    2565 generic.go:356] "Generic (PLEG): container finished" podID="bba7f867-311c-41a7-a02a-22badc2d7c38" containerID="3a193f12fbf12e585357ab79273dce7d26854d88de2aee1dc3ffed6817dedc3f" exitCode=0
Jul 22 12:03:56.716048 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:56.715667    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2wh82" event={"ID":"bba7f867-311c-41a7-a02a-22badc2d7c38","Type":"ContainerDied","Data":"3a193f12fbf12e585357ab79273dce7d26854d88de2aee1dc3ffed6817dedc3f"}
Jul 22 12:03:56.881375 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:56.881341    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:57.721404 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:57.721364    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2wh82" event={"ID":"bba7f867-311c-41a7-a02a-22badc2d7c38","Type":"ContainerStarted","Data":"285f24101875c61ba739700a1109bc3df34ef0d6f59488611801709101e94732"}
Jul 22 12:03:57.721888 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:57.721418    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2wh82" event={"ID":"bba7f867-311c-41a7-a02a-22badc2d7c38","Type":"ContainerStarted","Data":"627b7bc0250aa1b49157b84f85c573990394e1f7ca8e2937b75375078bbbd89b"}
Jul 22 12:03:57.746197 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:57.746146    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2wh82" podStartSLOduration=3.417624186 podStartE2EDuration="4.746130247s" totalImagesPullingTime="1.265415292s" totalInitContainerRuntime="63.090769ms" isStatefulPod=true podCreationTimestamp="2026-07-22 12:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:57.745861429 +0000 UTC m=+158.169292477" watchObservedRunningTime="2026-07-22 12:03:57.746130247 +0000 UTC m=+158.169561302"
Jul 22 12:03:57.917729 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:57.917676    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a"
Jul 22 12:03:58.005835 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:58.005729    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vmvx2" podUID="4f461d01-9d3e-4ba4-9dd7-8207c974e321"
Jul 22 12:03:58.117991 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.117959    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:58.137970 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:03:58.137936    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kfm2l" podUID="32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773"
Jul 22 12:03:58.198440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.198409    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:58.542170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.542132    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:58.641959 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.641925    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:58.724264 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.724233    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:03:58.724651 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.724233    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:03:58.865890 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.865812    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:03:58.930996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:03:58.930963    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:00.327410 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:00.327375    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:01.532268 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:01.532227    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:01.610644 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:01.610607    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:02.875338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.875305    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:02.877597 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.877576    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"image-registry-7f885957f8-tc8t5\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") " pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:02.931657 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.931630    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t9ltr\""
Jul 22 12:04:02.935722 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.935706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:02.976688 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.976636    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:04:02.978971 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:02.978948    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f461d01-9d3e-4ba4-9dd7-8207c974e321-metrics-tls\") pod \"dns-default-vmvx2\" (UID: \"4f461d01-9d3e-4ba4-9dd7-8207c974e321\") " pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:04:03.089538 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.089487    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:04:03.092294 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:04:03.092264    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda40d227_684c_48f5_97a5_0ac229ee4a8a.slice/crio-a026662ef3a843f60f1350a1fa370003092550f8e724d8868bb3ae1cf527b2b7 WatchSource:0}: Error finding container a026662ef3a843f60f1350a1fa370003092550f8e724d8868bb3ae1cf527b2b7: Status 404 returned error can't find the container with id a026662ef3a843f60f1350a1fa370003092550f8e724d8868bb3ae1cf527b2b7
Jul 22 12:04:03.221827 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.221766    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:03.228300 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.228280    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q9wgb\""
Jul 22 12:04:03.236159 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.236142    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:04:03.320376 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.320342    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:03.366164 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.366134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmvx2"]
Jul 22 12:04:03.368917 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:04:03.368889    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f461d01_9d3e_4ba4_9dd7_8207c974e321.slice/crio-0148f09c1f230a2b7869fade9c28c7fc127ae23f11452e2b01540d39dad0463a WatchSource:0}: Error finding container 0148f09c1f230a2b7869fade9c28c7fc127ae23f11452e2b01540d39dad0463a: Status 404 returned error can't find the container with id 0148f09c1f230a2b7869fade9c28c7fc127ae23f11452e2b01540d39dad0463a
Jul 22 12:04:03.741261 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.741222    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmvx2" event={"ID":"4f461d01-9d3e-4ba4-9dd7-8207c974e321","Type":"ContainerStarted","Data":"0148f09c1f230a2b7869fade9c28c7fc127ae23f11452e2b01540d39dad0463a"}
Jul 22 12:04:03.742667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.742642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" event={"ID":"da40d227-684c-48f5-97a5-0ac229ee4a8a","Type":"ContainerStarted","Data":"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"}
Jul 22 12:04:03.742828 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.742682    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" event={"ID":"da40d227-684c-48f5-97a5-0ac229ee4a8a","Type":"ContainerStarted","Data":"a026662ef3a843f60f1350a1fa370003092550f8e724d8868bb3ae1cf527b2b7"}
Jul 22 12:04:03.742828 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.742753    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:03.780897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:03.780851    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" podStartSLOduration=163.780835458 podStartE2EDuration="2m43.780835458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:03.779905997 +0000 UTC m=+164.203337046" watchObservedRunningTime="2026-07-22 12:04:03.780835458 +0000 UTC m=+164.204266513"
Jul 22 12:04:05.752472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:05.752436    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmvx2" event={"ID":"4f461d01-9d3e-4ba4-9dd7-8207c974e321","Type":"ContainerStarted","Data":"2a89e73df86cc1b12c49868647c01e995e78e7a62e62ad35436a1db9f9f3acc8"}
Jul 22 12:04:05.752927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:05.752519    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmvx2" event={"ID":"4f461d01-9d3e-4ba4-9dd7-8207c974e321","Type":"ContainerStarted","Data":"cdbde49dc2aceea004df94427a19cf790e45076eae5109e5cacbfb32096ec702"}
Jul 22 12:04:05.752927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:05.752561    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:04:05.778880 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:05.778831    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vmvx2" podStartSLOduration=129.961450529 podStartE2EDuration="2m11.778818139s" totalImagesPullingTime="1.81736761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:05.778429389 +0000 UTC m=+166.201860444" watchObservedRunningTime="2026-07-22 12:04:05.778818139 +0000 UTC m=+166.202249194"
Jul 22 12:04:06.025234 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:06.025204    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:04:06.196933 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:06.196894    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:06.322236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:06.322149    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:06.426832 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:06.426785    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:12.128467 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:12.128430    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:04:12.251291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:12.251240    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:12.355058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:12.355022    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:15.757503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:15.757471    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vmvx2"
Jul 22 12:04:19.427137 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:19.427099    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:19.524421 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:19.524380    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:21.612835 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:21.612800    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:21.718468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:21.718430    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:21.808404 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:21.808371    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:24.910294 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:24.910260    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:24.981334 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:24.981293    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:25.234735 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:25.234653    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:25.336095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:25.336062    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:25.439119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:25.439082    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:26.031868 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:26.031838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:27.305069 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:27.305036    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:27.372384 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:27.372351    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:31.044709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.044661    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a" containerName="registry" containerID="cri-o://e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1" gracePeriod=30
Jul 22 12:04:31.278504 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.278478    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:31.302113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.302113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302108    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.302296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302130    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.302296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302172    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.302382 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302291    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.302721 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302590    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:31.302914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302820    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:04:31.302914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302341    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.303042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302952    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.303042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.302995    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmfbk\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk\") pod \"da40d227-684c-48f5-97a5-0ac229ee4a8a\" (UID: \"da40d227-684c-48f5-97a5-0ac229ee4a8a\") "
Jul 22 12:04:31.303275 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.303193    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-certificates\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.303275 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.303213    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da40d227-684c-48f5-97a5-0ac229ee4a8a-trusted-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.305665 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.305613    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:04:31.305859 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.305833    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:04:31.306070 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.306025    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:04:31.306174 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.306073    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk" (OuterVolumeSpecName: "kube-api-access-lmfbk") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "kube-api-access-lmfbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:04:31.306174 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.306094    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:04:31.312397 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.312371    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "da40d227-684c-48f5-97a5-0ac229ee4a8a" (UID: "da40d227-684c-48f5-97a5-0ac229ee4a8a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:04:31.403562 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403525    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmfbk\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-kube-api-access-lmfbk\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.403562 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403557    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da40d227-684c-48f5-97a5-0ac229ee4a8a-ca-trust-extracted\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.403562 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403566    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-bound-sa-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.403783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403577    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-image-registry-private-configuration\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.403783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403587    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da40d227-684c-48f5-97a5-0ac229ee4a8a-registry-tls\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.403783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.403595    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da40d227-684c-48f5-97a5-0ac229ee4a8a-installation-pull-secrets\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:04:31.837316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.837282    2565 generic.go:356] "Generic (PLEG): container finished" podID="da40d227-684c-48f5-97a5-0ac229ee4a8a" containerID="e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1" exitCode=0
Jul 22 12:04:31.837493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.837329    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5"
Jul 22 12:04:31.837493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.837330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" event={"ID":"da40d227-684c-48f5-97a5-0ac229ee4a8a","Type":"ContainerDied","Data":"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"}
Jul 22 12:04:31.837493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.837443    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f885957f8-tc8t5" event={"ID":"da40d227-684c-48f5-97a5-0ac229ee4a8a","Type":"ContainerDied","Data":"a026662ef3a843f60f1350a1fa370003092550f8e724d8868bb3ae1cf527b2b7"}
Jul 22 12:04:31.837493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.837484    2565 scope.go:117] "RemoveContainer" containerID="e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"
Jul 22 12:04:31.846356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.846339    2565 scope.go:117] "RemoveContainer" containerID="e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"
Jul 22 12:04:31.846618 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:04:31.846598    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1\": container with ID starting with e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1 not found: ID does not exist" containerID="e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"
Jul 22 12:04:31.846667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.846624    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1"} err="failed to get container status \"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1\": rpc error: code = NotFound desc = could not find container \"e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1\": container with ID starting with e4656590b7b72cee600f973a8ca12fefe6576b1499683aa61369adc2364936b1 not found: ID does not exist"
Jul 22 12:04:31.858970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.858949    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:04:31.865209 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:31.865190    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f885957f8-tc8t5"]
Jul 22 12:04:32.132113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:32.132030    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a" path="/var/lib/kubelet/pods/da40d227-684c-48f5-97a5-0ac229ee4a8a/volumes"
Jul 22 12:04:42.543605 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:42.543572    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:04:42.611429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:04:42.611398    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:10.161681 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:10.161641    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:10.238322 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:10.238284    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:14.096750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:14.096703    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:14.163201 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:14.163166    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:22.114033 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:22.113978    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:22.174227 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:22.174194    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:24.865292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:24.865247    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:24.966050 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:24.966008    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:25.068063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:25.068021    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:27.223482 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:27.223448    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:27.323916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:27.323878    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:31.940234 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:31.940186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:05:31.942484 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:31.942456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773-metrics-certs\") pod \"network-metrics-daemon-kfm2l\" (UID: \"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773\") " pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:05:32.232062 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:32.231979    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4n945\""
Jul 22 12:05:32.239906 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:32.239888    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfm2l"
Jul 22 12:05:32.364103 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:32.364079    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfm2l"]
Jul 22 12:05:32.366444 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:05:32.366416    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cf26a5_20c4_42ff_b3b5_f5ebb2cf8773.slice/crio-95b30bb2f4bb1a6c1a45566faceabb23ed2c726a1c165e15819057dbd96603b3 WatchSource:0}: Error finding container 95b30bb2f4bb1a6c1a45566faceabb23ed2c726a1c165e15819057dbd96603b3: Status 404 returned error can't find the container with id 95b30bb2f4bb1a6c1a45566faceabb23ed2c726a1c165e15819057dbd96603b3
Jul 22 12:05:33.031626 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:33.031583    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfm2l" event={"ID":"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773","Type":"ContainerStarted","Data":"95b30bb2f4bb1a6c1a45566faceabb23ed2c726a1c165e15819057dbd96603b3"}
Jul 22 12:05:34.036337 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:34.036305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfm2l" event={"ID":"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773","Type":"ContainerStarted","Data":"5abdf3f0f4878c6c7a3f008f0d24381a5d92498bc954f5b1c4ac1e1e2d844d68"}
Jul 22 12:05:34.036703 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:34.036356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfm2l" event={"ID":"32cf26a5-20c4-42ff-b3b5-f5ebb2cf8773","Type":"ContainerStarted","Data":"a560bccc904695c350bfbcf674af2e5c33752d3142d7d247f1f8285f26051d43"}
Jul 22 12:05:34.159866 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:34.159837    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:34.234016 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:34.233989    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:54.525606 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:54.525524    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:54.621057 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:54.621026    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:57.463585 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:57.463544    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:05:58.860410 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:05:58.860368    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:13.925210 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:13.925170    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:20.048513 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:20.048483    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 12:06:20.886152 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:20.886113    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:20.988347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:20.988315    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:21.668694 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:21.668649    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:22.658095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:22.658057    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:25.477107 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:25.477072    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:25.579057 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:25.579020    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:28.268156 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:28.268118    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:28.359937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:28.359900    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:28.514768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:28.514734    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:29.553152 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:29.553116    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:30.678338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:30.678304    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:30.869051 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:30.869013    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:32.126238 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:32.126201    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:32.225587 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:32.225546    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:36.288977 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.288922    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kfm2l" podStartSLOduration=315.061320634 podStartE2EDuration="5m16.288905381s" totalImagesPullingTime="1.227584747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:05:34.054841289 +0000 UTC m=+254.478272344" watchObservedRunningTime="2026-07-22 12:06:36.288905381 +0000 UTC m=+316.712336445"
Jul 22 12:06:36.289609 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.289590    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-smnxg"]
Jul 22 12:06:36.289839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.289826    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a" containerName="registry"
Jul 22 12:06:36.289886 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.289843    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a" containerName="registry"
Jul 22 12:06:36.289923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.289912    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="da40d227-684c-48f5-97a5-0ac229ee4a8a" containerName="registry"
Jul 22 12:06:36.292616 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.292601    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.295744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.295725    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 22 12:06:36.296024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.296010    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hmqcz\""
Jul 22 12:06:36.296582 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.296566    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 22 12:06:36.306261 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.306240    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-smnxg"]
Jul 22 12:06:36.354937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.354905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.355095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.354944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xpz\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-kube-api-access-c9xpz\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.456283 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.456244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.456283 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.456285    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xpz\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-kube-api-access-c9xpz\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.465657 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.465626    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.465758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.465702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xpz\" (UniqueName: \"kubernetes.io/projected/92e66c5e-9aac-4f8b-a88a-b98475e48021-kube-api-access-c9xpz\") pod \"cert-manager-webhook-6486ffd997-smnxg\" (UID: \"92e66c5e-9aac-4f8b-a88a-b98475e48021\") " pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.601057 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.600963    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:36.727765 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.727735    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-smnxg"]
Jul 22 12:06:36.730846 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:06:36.730820    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92e66c5e_9aac_4f8b_a88a_b98475e48021.slice/crio-85e3f26eb047b9e2ce7c722d29ee18cd734b607c3c440ff50989b04213b31bb8 WatchSource:0}: Error finding container 85e3f26eb047b9e2ce7c722d29ee18cd734b607c3c440ff50989b04213b31bb8: Status 404 returned error can't find the container with id 85e3f26eb047b9e2ce7c722d29ee18cd734b607c3c440ff50989b04213b31bb8
Jul 22 12:06:36.733219 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:36.733203    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:06:37.230011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:37.229973    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg" event={"ID":"92e66c5e-9aac-4f8b-a88a-b98475e48021","Type":"ContainerStarted","Data":"85e3f26eb047b9e2ce7c722d29ee18cd734b607c3c440ff50989b04213b31bb8"}
Jul 22 12:06:38.479853 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:38.479749    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:40.057288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.057191    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:40.158412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.158375    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:40.243582 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.243551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg" event={"ID":"92e66c5e-9aac-4f8b-a88a-b98475e48021","Type":"ContainerStarted","Data":"a9c41b44900d8f359439fb6f5d343d947f93f24c4f82e96e34e78420bcb486eb"}
Jul 22 12:06:40.243829 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.243766    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:40.264189 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.264143    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg" podStartSLOduration=1.195961193 podStartE2EDuration="4.264129628s" totalImagesPullingTime="3.068168435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:06:40.263430794 +0000 UTC m=+320.686861848" watchObservedRunningTime="2026-07-22 12:06:40.264129628 +0000 UTC m=+320.687560683"
Jul 22 12:06:40.386077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.385996    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:40.484883 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:40.484832    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:46.249515 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:46.249481    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-smnxg"
Jul 22 12:06:47.035197 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:47.035158    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:47.163556 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:47.163514    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:06:49.159269 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:06:49.159236    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:01.449365 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:01.449327    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:01.898053 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:01.898009    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:01.997149 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:01.997113    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:02.526539 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:02.526502    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.147373 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.147342    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.258861 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.258828    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.315042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.315011    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"]
Jul 22 12:07:03.318007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.317986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.338034 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.338013    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"]
Jul 22 12:07:03.338361 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.338348    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-47874\""
Jul 22 12:07:03.338914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.338901    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 12:07:03.339198 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.339179    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 12:07:03.339306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.339183    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 12:07:03.339369 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.339313    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 12:07:03.447975 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.447887    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.447975 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.447925    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tdc\" (UniqueName: \"kubernetes.io/projected/d498609e-1203-469d-a9b0-4ea8f34ccb14-kube-api-access-x7tdc\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.447975 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.447949    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.448229 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.448041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.448229 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.448125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.548972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.548926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.549451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.548986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.549451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.549012    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tdc\" (UniqueName: \"kubernetes.io/projected/d498609e-1203-469d-a9b0-4ea8f34ccb14-kube-api-access-x7tdc\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.549451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.549041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.549451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.549068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.549620 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.549452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-manifests\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.551244 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.551219    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d498609e-1203-469d-a9b0-4ea8f34ccb14-tmp\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.551359 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.551276    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-webhook-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.551422 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.551399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d498609e-1203-469d-a9b0-4ea8f34ccb14-apiservice-cert\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.561850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.561821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tdc\" (UniqueName: \"kubernetes.io/projected/d498609e-1203-469d-a9b0-4ea8f34ccb14-kube-api-access-x7tdc\") pod \"opendatahub-operator-controller-manager-979495d87-bgn22\" (UID: \"d498609e-1203-469d-a9b0-4ea8f34ccb14\") " pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.626972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.626941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:03.684436 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.684399    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.760570 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.760547    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"]
Jul 22 12:07:03.762335 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:03.762313    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd498609e_1203_469d_a9b0_4ea8f34ccb14.slice/crio-e553c68709a4db6b8c0365a59e9526e52b98cd51b889cc1c485b92344d5a29e9 WatchSource:0}: Error finding container e553c68709a4db6b8c0365a59e9526e52b98cd51b889cc1c485b92344d5a29e9: Status 404 returned error can't find the container with id e553c68709a4db6b8c0365a59e9526e52b98cd51b889cc1c485b92344d5a29e9
Jul 22 12:07:03.768450 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.768427    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.851661 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.851617    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:03.949285 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:03.949249    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:04.054516 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:04.054483    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:04.323085 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:04.322999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22" event={"ID":"d498609e-1203-469d-a9b0-4ea8f34ccb14","Type":"ContainerStarted","Data":"e553c68709a4db6b8c0365a59e9526e52b98cd51b889cc1c485b92344d5a29e9"}
Jul 22 12:07:06.057448 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:06.055615    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:07.336173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:07.336125    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22" event={"ID":"d498609e-1203-469d-a9b0-4ea8f34ccb14","Type":"ContainerStarted","Data":"fd52da4346c0b083734961b392ed5ac0aa801947c15649f77507af177acaa30d"}
Jul 22 12:07:08.342680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:08.342598    2565 generic.go:356] "Generic (PLEG): container finished" podID="d498609e-1203-469d-a9b0-4ea8f34ccb14" containerID="fd52da4346c0b083734961b392ed5ac0aa801947c15649f77507af177acaa30d" exitCode=0
Jul 22 12:07:08.342680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:08.342642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22" event={"ID":"d498609e-1203-469d-a9b0-4ea8f34ccb14","Type":"ContainerDied","Data":"fd52da4346c0b083734961b392ed5ac0aa801947c15649f77507af177acaa30d"}
Jul 22 12:07:09.347912 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:09.347876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22" event={"ID":"d498609e-1203-469d-a9b0-4ea8f34ccb14","Type":"ContainerStarted","Data":"04b08e9ffaa6905dcbdda5861d841a55b28e8be3b83996d868daa098f917b772"}
Jul 22 12:07:09.348284 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:09.347991    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:09.374326 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:09.374271    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22" podStartSLOduration=1.75152336 podStartE2EDuration="6.37425593s" totalImagesPullingTime="3.528949284s" totalInitContainerRuntime="1.093783286s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:09.372354207 +0000 UTC m=+349.795785262" watchObservedRunningTime="2026-07-22 12:07:09.37425593 +0000 UTC m=+349.797686985"
Jul 22 12:07:14.945547 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:14.945467    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:15.045727 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:15.045691    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:20.353878 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:20.353848    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-979495d87-bgn22"
Jul 22 12:07:24.320565 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:24.320515    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:24.455698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:24.455663    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:25.151639 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.151600    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqnk8"]
Jul 22 12:07:25.158637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.158609    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.161396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.161373    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 12:07:25.161491 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.161373    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9xrt7\""
Jul 22 12:07:25.174610 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.174588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqnk8"]
Jul 22 12:07:25.217419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.217385    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:25.312562 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.312526    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqsfk\" (UniqueName: \"kubernetes.io/projected/ab892338-e44c-454c-b5e8-ba3b26f723e8-kube-api-access-rqsfk\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.312731 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.312575    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.319914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.319879    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:25.413350 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.413269    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqsfk\" (UniqueName: \"kubernetes.io/projected/ab892338-e44c-454c-b5e8-ba3b26f723e8-kube-api-access-rqsfk\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.413350 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.413318    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.413863 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:25.413423    2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 22 12:07:25.413863 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:25.413488    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert podName:ab892338-e44c-454c-b5e8-ba3b26f723e8 nodeName:}" failed. No retries permitted until 2026-07-22 12:07:25.913468372 +0000 UTC m=+366.336899413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert") pod "odh-model-controller-7998c67dc4-nqnk8" (UID: "ab892338-e44c-454c-b5e8-ba3b26f723e8") : secret "odh-model-controller-webhook-cert" not found
Jul 22 12:07:25.424448 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.424427    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqsfk\" (UniqueName: \"kubernetes.io/projected/ab892338-e44c-454c-b5e8-ba3b26f723e8-kube-api-access-rqsfk\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.610648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.610610    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:25.917237 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.917195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:25.919526 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:25.919507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab892338-e44c-454c-b5e8-ba3b26f723e8-cert\") pod \"odh-model-controller-7998c67dc4-nqnk8\" (UID: \"ab892338-e44c-454c-b5e8-ba3b26f723e8\") " pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:26.068683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.068633    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:26.148585 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.148558    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:26.155424 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.155393    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z55d4"]
Jul 22 12:07:26.158813 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.158774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.162266 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.162241    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 22 12:07:26.162390 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.162247    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6gqgd\""
Jul 22 12:07:26.174426 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.174369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z55d4"]
Jul 22 12:07:26.205885 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.205857    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nqnk8"]
Jul 22 12:07:26.208956 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:26.208932    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab892338_e44c_454c_b5e8_ba3b26f723e8.slice/crio-baeb3a627e462a62a096160cac4ad3c98151dd05ffe076ba448419d09b6222c1 WatchSource:0}: Error finding container baeb3a627e462a62a096160cac4ad3c98151dd05ffe076ba448419d09b6222c1: Status 404 returned error can't find the container with id baeb3a627e462a62a096160cac4ad3c98151dd05ffe076ba448419d09b6222c1
Jul 22 12:07:26.320362 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.320327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m5rm\" (UniqueName: \"kubernetes.io/projected/32a38fff-3a9c-4434-befc-9a776b417a17-kube-api-access-8m5rm\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.320521 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.320383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.405599 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.405562    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8" event={"ID":"ab892338-e44c-454c-b5e8-ba3b26f723e8","Type":"ContainerStarted","Data":"baeb3a627e462a62a096160cac4ad3c98151dd05ffe076ba448419d09b6222c1"}
Jul 22 12:07:26.421008 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.420977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.421345 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.421033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m5rm\" (UniqueName: \"kubernetes.io/projected/32a38fff-3a9c-4434-befc-9a776b417a17-kube-api-access-8m5rm\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.421345 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:26.421117    2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 22 12:07:26.421345 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:26.421178    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs podName:32a38fff-3a9c-4434-befc-9a776b417a17 nodeName:}" failed. No retries permitted until 2026-07-22 12:07:26.921157634 +0000 UTC m=+367.344588667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs") pod "model-serving-api-54d89bd5fc-z55d4" (UID: "32a38fff-3a9c-4434-befc-9a776b417a17") : secret "model-serving-api-tls" not found
Jul 22 12:07:26.434010 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.433945    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m5rm\" (UniqueName: \"kubernetes.io/projected/32a38fff-3a9c-4434-befc-9a776b417a17-kube-api-access-8m5rm\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.787004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.786974    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:26.925802 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.925755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:26.928148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:26.928121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32a38fff-3a9c-4434-befc-9a776b417a17-tls-certs\") pod \"model-serving-api-54d89bd5fc-z55d4\" (UID: \"32a38fff-3a9c-4434-befc-9a776b417a17\") " pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:27.070102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:27.070004    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:27.198554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:27.198522    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z55d4"]
Jul 22 12:07:27.201656 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:27.201630    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a38fff_3a9c_4434_befc_9a776b417a17.slice/crio-bce2ef65e6e13915ee3b6fdf1d3f5a72673ddd4a2c1fcb26a88069fad57680b5 WatchSource:0}: Error finding container bce2ef65e6e13915ee3b6fdf1d3f5a72673ddd4a2c1fcb26a88069fad57680b5: Status 404 returned error can't find the container with id bce2ef65e6e13915ee3b6fdf1d3f5a72673ddd4a2c1fcb26a88069fad57680b5
Jul 22 12:07:27.410322 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:27.410238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4" event={"ID":"32a38fff-3a9c-4434-befc-9a776b417a17","Type":"ContainerStarted","Data":"bce2ef65e6e13915ee3b6fdf1d3f5a72673ddd4a2c1fcb26a88069fad57680b5"}
Jul 22 12:07:29.419054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:29.419016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4" event={"ID":"32a38fff-3a9c-4434-befc-9a776b417a17","Type":"ContainerStarted","Data":"c9053a41a8d838156c76fbf224d6269ebf64ae91e846af7bd450ac5ace7469f6"}
Jul 22 12:07:29.419438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:29.419145    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:29.440336 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:29.440290    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4" podStartSLOduration=1.816464372 podStartE2EDuration="3.440275917s" totalImagesPullingTime="1.623811545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:29.439320665 +0000 UTC m=+369.862751721" watchObservedRunningTime="2026-07-22 12:07:29.440275917 +0000 UTC m=+369.863706971"
Jul 22 12:07:30.128218 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.128177    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:30.229140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.229106    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"]
Jul 22 12:07:30.232506 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.232477    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.235232 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.234836    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 12:07:30.236570 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.236536    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 12:07:30.236687 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.236542    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 12:07:30.236864 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.236842    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 12:07:30.236864 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.236857    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-p5pj9\""
Jul 22 12:07:30.246023 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.245980    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"]
Jul 22 12:07:30.250568 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.250548    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:30.352554 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.352517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tls-certs\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.352737 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.352565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tmp\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.352737 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.352607    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s2xl\" (UniqueName: \"kubernetes.io/projected/b51d5e0f-781c-46dd-b02a-8e137e7a7874-kube-api-access-2s2xl\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.453726 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.453644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tls-certs\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.453726 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.453684    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tmp\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.454252 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.453729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s2xl\" (UniqueName: \"kubernetes.io/projected/b51d5e0f-781c-46dd-b02a-8e137e7a7874-kube-api-access-2s2xl\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.456273 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.456243    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tmp\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.456452 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.456432    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51d5e0f-781c-46dd-b02a-8e137e7a7874-tls-certs\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.464222 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.464175    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s2xl\" (UniqueName: \"kubernetes.io/projected/b51d5e0f-781c-46dd-b02a-8e137e7a7874-kube-api-access-2s2xl\") pod \"kube-auth-proxy-f68f699cd-vtdjv\" (UID: \"b51d5e0f-781c-46dd-b02a-8e137e7a7874\") " pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:30.544870 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:30.544838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"
Jul 22 12:07:31.696932 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:31.696906    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv"]
Jul 22 12:07:31.697637 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:31.697610    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb51d5e0f_781c_46dd_b02a_8e137e7a7874.slice/crio-256afff5411c90337c70c91982cdaffe2dadd39926ed82b109afaf5438ad0ee9 WatchSource:0}: Error finding container 256afff5411c90337c70c91982cdaffe2dadd39926ed82b109afaf5438ad0ee9: Status 404 returned error can't find the container with id 256afff5411c90337c70c91982cdaffe2dadd39926ed82b109afaf5438ad0ee9
Jul 22 12:07:31.951768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:31.951684    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:32.432770 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:32.432733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8" event={"ID":"ab892338-e44c-454c-b5e8-ba3b26f723e8","Type":"ContainerStarted","Data":"d24aae49d8c19bdb4e1ae4ff372bd948819ca92d35875ead2d0b891b6cf2c4b3"}
Jul 22 12:07:32.432992 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:32.432884    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:32.434031 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:32.434001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv" event={"ID":"b51d5e0f-781c-46dd-b02a-8e137e7a7874","Type":"ContainerStarted","Data":"256afff5411c90337c70c91982cdaffe2dadd39926ed82b109afaf5438ad0ee9"}
Jul 22 12:07:32.458840 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:32.458766    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8" podStartSLOduration=2.046710734 podStartE2EDuration="7.458747548s" totalImagesPullingTime="5.412036814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:32.456685875 +0000 UTC m=+372.880116930" watchObservedRunningTime="2026-07-22 12:07:32.458747548 +0000 UTC m=+372.882178604"
Jul 22 12:07:33.947802 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.947755    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"]
Jul 22 12:07:33.952101 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.952076    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:33.954408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.954388    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 12:07:33.954537 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.954517    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 12:07:33.954608 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.954553    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 12:07:33.954691 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.954678    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 12:07:33.954908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.954891    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zmczj\""
Jul 22 12:07:33.984389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:33.984366    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"]
Jul 22 12:07:34.082926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.082843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.082926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.082876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.082926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.082900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.082926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.082920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.083157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.082982    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2pdz\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-kube-api-access-j2pdz\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.083157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.083051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.083157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.083085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/962303b0-f3b0-40be-82af-979be562ac6c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183729 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183699    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183729 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183817    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2pdz\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-kube-api-access-j2pdz\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.183978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.183867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/962303b0-f3b0-40be-82af-979be562ac6c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.184633 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.184579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.186184 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.186152    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/962303b0-f3b0-40be-82af-979be562ac6c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.186291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.186233    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.186549 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.186530    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.186612 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.186549    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/962303b0-f3b0-40be-82af-979be562ac6c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.194557 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.194536    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.194640 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.194611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2pdz\" (UniqueName: \"kubernetes.io/projected/962303b0-f3b0-40be-82af-979be562ac6c-kube-api-access-j2pdz\") pod \"istiod-openshift-gateway-94bb8fbfd-tjts8\" (UID: \"962303b0-f3b0-40be-82af-979be562ac6c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.260591 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.260557    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:34.397049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.397018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"]
Jul 22 12:07:34.400462 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:34.400430    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962303b0_f3b0_40be_82af_979be562ac6c.slice/crio-e064069bff7ac46c775f262610fed603c0aa81c521e69dfcbe4e205293c35928 WatchSource:0}: Error finding container e064069bff7ac46c775f262610fed603c0aa81c521e69dfcbe4e205293c35928: Status 404 returned error can't find the container with id e064069bff7ac46c775f262610fed603c0aa81c521e69dfcbe4e205293c35928
Jul 22 12:07:34.443179 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.443143    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8" event={"ID":"962303b0-f3b0-40be-82af-979be562ac6c","Type":"ContainerStarted","Data":"e064069bff7ac46c775f262610fed603c0aa81c521e69dfcbe4e205293c35928"}
Jul 22 12:07:34.444455 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:34.444435    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv" event={"ID":"b51d5e0f-781c-46dd-b02a-8e137e7a7874","Type":"ContainerStarted","Data":"e6b0e77780fa40dfdb4c58db41ad962ad1deb584e087518d72bf90638cfe0261"}
Jul 22 12:07:36.613602 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.613569    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:36.732844 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.732802    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:36.750415 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.750372    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f68f699cd-vtdjv" podStartSLOduration=4.65579982 podStartE2EDuration="6.75035561s" totalImagesPullingTime="2.09455579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:34.47419067 +0000 UTC m=+374.897621726" watchObservedRunningTime="2026-07-22 12:07:36.75035561 +0000 UTC m=+377.173786703"
Jul 22 12:07:36.750567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.750489    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x5744"]
Jul 22 12:07:36.753458 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.753440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:36.756736 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.756713    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8j2jq\""
Jul 22 12:07:36.757358 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.757340    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 22 12:07:36.762002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.761971    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:36.762075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.762030    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:36.776079 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.776055    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"]
Jul 22 12:07:36.780341 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.780320    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:36.784119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.783672    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-h2qc6\""
Jul 22 12:07:36.784119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.783702    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 22 12:07:36.784119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.784050    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x5744"]
Jul 22 12:07:36.797811 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.797762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"]
Jul 22 12:07:36.848926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.848877    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:36.908288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.908203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7v8p\" (UniqueName: \"kubernetes.io/projected/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-kube-api-access-s7v8p\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:36.908288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.908248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:36.908499 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.908337    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2l4m\" (UniqueName: \"kubernetes.io/projected/0c985ceb-d49e-4073-98c2-07687d2cfaa0-kube-api-access-p2l4m\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:36.908499 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.908403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:36.945910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:36.945874    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:37.009246 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.009204    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2l4m\" (UniqueName: \"kubernetes.io/projected/0c985ceb-d49e-4073-98c2-07687d2cfaa0-kube-api-access-p2l4m\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.009429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.009267    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.009429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.009306    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7v8p\" (UniqueName: \"kubernetes.io/projected/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-kube-api-access-s7v8p\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:37.009429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.009341    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:37.009590 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:37.009431    2565 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 22 12:07:37.009590 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:07:37.009502    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert podName:0c985ceb-d49e-4073-98c2-07687d2cfaa0 nodeName:}" failed. No retries permitted until 2026-07-22 12:07:37.509482173 +0000 UTC m=+377.932913206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert") pod "llmisvc-controller-manager-777c6bd7d5-lp4nc" (UID: "0c985ceb-d49e-4073-98c2-07687d2cfaa0") : secret "llmisvc-webhook-server-cert" not found
Jul 22 12:07:37.011745 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.011721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-cert\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:37.022510 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.022484    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7v8p\" (UniqueName: \"kubernetes.io/projected/d7825f8b-1508-4efa-a3a1-dde290b0a2e7-kube-api-access-s7v8p\") pod \"kserve-controller-manager-d6f4b65b8-x5744\" (UID: \"d7825f8b-1508-4efa-a3a1-dde290b0a2e7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:37.022868 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.022850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2l4m\" (UniqueName: \"kubernetes.io/projected/0c985ceb-d49e-4073-98c2-07687d2cfaa0-kube-api-access-p2l4m\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.063738 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.063715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:37.215990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.215911    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x5744"]
Jul 22 12:07:37.220659 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:37.220635    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7825f8b_1508_4efa_a3a1_dde290b0a2e7.slice/crio-0b63e0ed33c15e6015dc394a0b9030a97d8c13771caeae5af8f06622402509c4 WatchSource:0}: Error finding container 0b63e0ed33c15e6015dc394a0b9030a97d8c13771caeae5af8f06622402509c4: Status 404 returned error can't find the container with id 0b63e0ed33c15e6015dc394a0b9030a97d8c13771caeae5af8f06622402509c4
Jul 22 12:07:37.266779 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.266746    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:37.458973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.458936    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744" event={"ID":"d7825f8b-1508-4efa-a3a1-dde290b0a2e7","Type":"ContainerStarted","Data":"0b63e0ed33c15e6015dc394a0b9030a97d8c13771caeae5af8f06622402509c4"}
Jul 22 12:07:37.460397 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.460372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8" event={"ID":"962303b0-f3b0-40be-82af-979be562ac6c","Type":"ContainerStarted","Data":"1068eefd85d0caeb2ea860b33ac16a939b5a755fe36e8c84a851d63da05161b3"}
Jul 22 12:07:37.460582 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.460564    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:37.462136 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.462105    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-tjts8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:07:37.462229 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.462165    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8" podUID="962303b0-f3b0-40be-82af-979be562ac6c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:07:37.514090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.514008    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.516904 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.516876    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c985ceb-d49e-4073-98c2-07687d2cfaa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lp4nc\" (UID: \"0c985ceb-d49e-4073-98c2-07687d2cfaa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.543419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.543360    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8" podStartSLOduration=2.184062475 podStartE2EDuration="4.543342819s" totalImagesPullingTime="2.359280344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:37.541992705 +0000 UTC m=+377.965423761" watchObservedRunningTime="2026-07-22 12:07:37.543342819 +0000 UTC m=+377.966773872"
Jul 22 12:07:37.574089 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.574051    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:37.673934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.673896    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:37.704879 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.704841    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:37.854353 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.853970    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"]
Jul 22 12:07:37.856911 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:37.856879    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c985ceb_d49e_4073_98c2_07687d2cfaa0.slice/crio-af4a2f2a8c884a735bb824ab18148c764aa4c70f58b6875c256f00c624f20f6b WatchSource:0}: Error finding container af4a2f2a8c884a735bb824ab18148c764aa4c70f58b6875c256f00c624f20f6b: Status 404 returned error can't find the container with id af4a2f2a8c884a735bb824ab18148c764aa4c70f58b6875c256f00c624f20f6b
Jul 22 12:07:37.981212 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:37.981157    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:38.095232 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:38.095130    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:38.194108 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:38.194068    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:38.465542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:38.465467    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc" event={"ID":"0c985ceb-d49e-4073-98c2-07687d2cfaa0","Type":"ContainerStarted","Data":"af4a2f2a8c884a735bb824ab18148c764aa4c70f58b6875c256f00c624f20f6b"}
Jul 22 12:07:38.466164 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:38.466140    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-tjts8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:07:38.466282 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:38.466197    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8" podUID="962303b0-f3b0-40be-82af-979be562ac6c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:07:39.414573 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.414529    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"]
Jul 22 12:07:39.418361 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.418339    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.422771 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.422739    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rnqm5\""
Jul 22 12:07:39.434038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.434011    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"]
Jul 22 12:07:39.439837 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.439783    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:39.532224 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532233    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532317    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07dc49e4-8a85-4d5a-a800-494cb4701627-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532359    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532396    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr2g9\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-kube-api-access-kr2g9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.532823 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.532649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.552777 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.552742    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:39.634082 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634269 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634154    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634269 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634269 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634222    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07dc49e4-8a85-4d5a-a800-494cb4701627-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634580    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr2g9\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-kube-api-access-kr2g9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.634818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634653    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.635119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634879    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.635119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.635119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.634974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07dc49e4-8a85-4d5a-a800-494cb4701627-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.635119 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.635002    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.637132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.637097    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.637282 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.637259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.650654 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.650629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr2g9\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-kube-api-access-kr2g9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.652141 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.652120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07dc49e4-8a85-4d5a-a800-494cb4701627-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jv62k\" (UID: \"07dc49e4-8a85-4d5a-a800-494cb4701627\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.674495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.674403    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:39.730475 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.730442    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:39.957215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:39.957114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"]
Jul 22 12:07:39.959664 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:07:39.959624    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07dc49e4_8a85_4d5a_a800_494cb4701627.slice/crio-a1c7126906a50cda1cf29b67ff7bb15567983c01ad76b3bb715d34726192bf88 WatchSource:0}: Error finding container a1c7126906a50cda1cf29b67ff7bb15567983c01ad76b3bb715d34726192bf88: Status 404 returned error can't find the container with id a1c7126906a50cda1cf29b67ff7bb15567983c01ad76b3bb715d34726192bf88
Jul 22 12:07:40.426707 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:40.426670    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-z55d4"
Jul 22 12:07:40.479143 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:40.479106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744" event={"ID":"d7825f8b-1508-4efa-a3a1-dde290b0a2e7","Type":"ContainerStarted","Data":"967779675bdc4f390492e519e7a77a3302a1535fec0e2d318e4ece56dff2e36a"}
Jul 22 12:07:40.479312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:40.479198    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:07:40.480331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:40.480308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k" event={"ID":"07dc49e4-8a85-4d5a-a800-494cb4701627","Type":"ContainerStarted","Data":"a1c7126906a50cda1cf29b67ff7bb15567983c01ad76b3bb715d34726192bf88"}
Jul 22 12:07:41.466541 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:41.466504    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tjts8"
Jul 22 12:07:41.498445 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:41.498396    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744" podStartSLOduration=2.9538579130000002 podStartE2EDuration="5.498379745s" totalImagesPullingTime="2.544521832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:40.505703905 +0000 UTC m=+380.929134960" watchObservedRunningTime="2026-07-22 12:07:41.498379745 +0000 UTC m=+381.921810799"
Jul 22 12:07:42.291322 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:42.291283    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:42.364798 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:42.364752    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:42.734692 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:42.734652    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:42.735045 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:42.734727    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:42.735045 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:42.734767    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:07:43.441160 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.441133    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nqnk8"
Jul 22 12:07:43.497014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.496741    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc" event={"ID":"0c985ceb-d49e-4073-98c2-07687d2cfaa0","Type":"ContainerStarted","Data":"f6778c17fcd1ccacc9b475444378a5fbc8712a9a8e7013fe04c08e9fc24ab59e"}
Jul 22 12:07:43.497014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.497001    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:07:43.498690 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.498661    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k" event={"ID":"07dc49e4-8a85-4d5a-a800-494cb4701627","Type":"ContainerStarted","Data":"76ca5a73a8b36c8eec6b0740887d2395932fddb72d3f97d8eeaaa1eaa4234189"}
Jul 22 12:07:43.528131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.528072    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc" podStartSLOduration=2.096488301 podStartE2EDuration="7.528050812s" totalImagesPullingTime="5.431562511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:43.527543256 +0000 UTC m=+383.950974331" watchObservedRunningTime="2026-07-22 12:07:43.528050812 +0000 UTC m=+383.951481868"
Jul 22 12:07:43.563604 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.563543    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k" podStartSLOduration=1.790880485 podStartE2EDuration="4.563523099s" totalImagesPullingTime="2.772642614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:07:43.562674249 +0000 UTC m=+383.986105304" watchObservedRunningTime="2026-07-22 12:07:43.563523099 +0000 UTC m=+383.986954155"
Jul 22 12:07:43.730762 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:43.730723    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:44.736537 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:44.736506    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:44.891710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:44.891672    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:44.987831 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:44.987716    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:07:45.507439 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:45.507403    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:07:45.508267 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:07:45.508248    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jv62k"
Jul 22 12:08:11.491060 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:11.491025    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x5744"
Jul 22 12:08:14.505102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:14.505065    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lp4nc"
Jul 22 12:08:26.286843 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:26.286808    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:08:26.382939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:26.382900    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:08:26.522339 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:26.522308    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:08:55.785622 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:55.785584    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:08:56.775867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:56.775828    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:08:56.865147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:08:56.865114    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:00.815980 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:00.815947    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:03.228232 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:03.228188    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:03.599947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:03.599910    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:05.687080 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:05.687008    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:05.809267 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:05.809233    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:08.277077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:08.277038    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:10.836012 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:10.835976    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:10.955974 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:10.955936    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:11.100131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:11.100033    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:11.227725 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:11.227690    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:12.391045 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:12.391001    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:12.496139 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:12.496091    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:15.087677 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:15.087644    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:15.230910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:15.230873    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:17.849316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:17.849283    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:17.939764 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:17.939730    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:18.718527 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:18.718493    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:19.153899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:19.153864    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:22.623970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:22.623933    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:22.751363 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:22.751327    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:23.241091 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:23.241055    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:23.351103 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:23.351059    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:29.253685 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.253652    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"]
Jul 22 12:09:29.256852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.256834    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:29.260148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.260124    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 12:09:29.260479 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.260462    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 12:09:29.261017 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.260992    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jzwl2\""
Jul 22 12:09:29.279651 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.279623    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"]
Jul 22 12:09:29.295406 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.295378    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8lb8\" (UniqueName: \"kubernetes.io/projected/b42a81f1-0157-4aa5-bf2d-e54ac53cb868-kube-api-access-j8lb8\") pod \"limitador-operator-controller-manager-859d7b78bd-59lpb\" (UID: \"b42a81f1-0157-4aa5-bf2d-e54ac53cb868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:29.396573 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.396537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8lb8\" (UniqueName: \"kubernetes.io/projected/b42a81f1-0157-4aa5-bf2d-e54ac53cb868-kube-api-access-j8lb8\") pod \"limitador-operator-controller-manager-859d7b78bd-59lpb\" (UID: \"b42a81f1-0157-4aa5-bf2d-e54ac53cb868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:29.417205 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.417171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8lb8\" (UniqueName: \"kubernetes.io/projected/b42a81f1-0157-4aa5-bf2d-e54ac53cb868-kube-api-access-j8lb8\") pod \"limitador-operator-controller-manager-859d7b78bd-59lpb\" (UID: \"b42a81f1-0157-4aa5-bf2d-e54ac53cb868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:29.567145 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.567107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:29.702753 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:09:29.702704    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42a81f1_0157_4aa5_bf2d_e54ac53cb868.slice/crio-a25118fcc3201663322c5db5271c419f4c488a709094a4966804b6f2af1ab352 WatchSource:0}: Error finding container a25118fcc3201663322c5db5271c419f4c488a709094a4966804b6f2af1ab352: Status 404 returned error can't find the container with id a25118fcc3201663322c5db5271c419f4c488a709094a4966804b6f2af1ab352
Jul 22 12:09:29.705118 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.705099    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"]
Jul 22 12:09:29.924876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:29.924780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb" event={"ID":"b42a81f1-0157-4aa5-bf2d-e54ac53cb868","Type":"ContainerStarted","Data":"a25118fcc3201663322c5db5271c419f4c488a709094a4966804b6f2af1ab352"}
Jul 22 12:09:30.712459 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:30.712427    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:30.811219 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:30.811152    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:31.289678 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:31.289639    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:31.399546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:31.399497    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:32.938835 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:32.938779    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb" event={"ID":"b42a81f1-0157-4aa5-bf2d-e54ac53cb868","Type":"ContainerStarted","Data":"b664eb1cdce897a1935b6eea1b9cb4729fb8ac0c3ad6bb4ec2cb02a53593d91e"}
Jul 22 12:09:32.939253 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:32.938869    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:32.965919 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:32.965871    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb" podStartSLOduration=1.5829572920000001 podStartE2EDuration="3.965856001s" totalImagesPullingTime="2.382898709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:09:32.964741941 +0000 UTC m=+493.388172995" watchObservedRunningTime="2026-07-22 12:09:32.965856001 +0000 UTC m=+493.389287056"
Jul 22 12:09:34.873469 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:34.873428    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:35.016405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:35.016371    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:37.052875 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:37.052834    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:37.153046 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:37.153015    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:41.734486 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:41.734453    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:41.820945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:41.820904    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:43.944680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:43.944646    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-59lpb"
Jul 22 12:09:58.060491 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:58.060440    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:09:58.799459 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:09:58.799415    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:02.822535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:02.822498    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:03.142802 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.142689    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:03.241147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.241111    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:03.243350 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.243324    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:10:03.248440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.248422    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.251470 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.251445    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 12:10:03.251578 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.251499    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l7xw8\""
Jul 22 12:10:03.259654 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.259633    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:10:03.298850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.298811    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:10:03.383098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.383061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn886\" (UniqueName: \"kubernetes.io/projected/964dc4e7-fb39-4286-8831-48fe9214ca78-kube-api-access-dn886\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.383289 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.383122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/964dc4e7-fb39-4286-8831-48fe9214ca78-config-file\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.483982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.483899    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn886\" (UniqueName: \"kubernetes.io/projected/964dc4e7-fb39-4286-8831-48fe9214ca78-kube-api-access-dn886\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.483982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.483955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/964dc4e7-fb39-4286-8831-48fe9214ca78-config-file\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.484499 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.484481    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/964dc4e7-fb39-4286-8831-48fe9214ca78-config-file\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.493468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.493438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn886\" (UniqueName: \"kubernetes.io/projected/964dc4e7-fb39-4286-8831-48fe9214ca78-kube-api-access-dn886\") pod \"limitador-limitador-54d6d988d5-2d5xw\" (UID: \"964dc4e7-fb39-4286-8831-48fe9214ca78\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.558503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.558470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:03.698872 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.698848    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:10:03.701355 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:10:03.701325    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964dc4e7_fb39_4286_8831_48fe9214ca78.slice/crio-2bd7c42e48617e68d44253358f5249e887ce5248ec331ff8004d411c11276e77 WatchSource:0}: Error finding container 2bd7c42e48617e68d44253358f5249e887ce5248ec331ff8004d411c11276e77: Status 404 returned error can't find the container with id 2bd7c42e48617e68d44253358f5249e887ce5248ec331ff8004d411c11276e77
Jul 22 12:10:03.770939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.770906    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:03.857694 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:03.857655    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:04.066453 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:04.066376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw" event={"ID":"964dc4e7-fb39-4286-8831-48fe9214ca78","Type":"ContainerStarted","Data":"2bd7c42e48617e68d44253358f5249e887ce5248ec331ff8004d411c11276e77"}
Jul 22 12:10:07.082393 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:07.082355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw" event={"ID":"964dc4e7-fb39-4286-8831-48fe9214ca78","Type":"ContainerStarted","Data":"dd121fa1533b9387eaf8a479426c7525831c8809a4097753eebcc59231adcaa7"}
Jul 22 12:10:07.082768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:07.082478    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:07.108849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:07.108800    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw" podStartSLOduration=1.578271615 podStartE2EDuration="4.108772795s" totalImagesPullingTime="2.53050118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:10:07.107036526 +0000 UTC m=+527.530467581" watchObservedRunningTime="2026-07-22 12:10:07.108772795 +0000 UTC m=+527.532203861"
Jul 22 12:10:18.087683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:18.087605    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"
Jul 22 12:10:24.041101 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:24.041060    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:24.137415 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:24.137382    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:24.731918 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:24.731884    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:37.004947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:37.004911    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:37.134461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:37.134430    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:40.831368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:40.831330    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:44.922053 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:44.922001    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:45.089125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:45.089071    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:45.188503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:45.188410    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:45.485168 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:45.485069    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:48.382289 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:48.382234    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:10:48.489443 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:10:48.489402    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:19.055597 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:19.055565    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:19.155663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:19.155624    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:19.711005 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:19.710964    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:19.804392 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:19.804357    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:20.570255 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:20.570207    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:20.945293 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:20.945148    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:11:21.142074 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:21.142034    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:30.000143 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:30.000106    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:11:30.095242 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:11:30.095200    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:02.237454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:02.237416    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:02.881429 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:02.881394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:12:06.781599 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:06.781562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:13.495803 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:13.495745    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:14.152173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:14.152131    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:14.911253 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:14.911217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:12:26.048403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:26.048368    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:27.354408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:27.354378    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:12:29.146724 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:29.146688    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:39.847277 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:39.847241    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:41.608090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:41.608055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:12:44.719955 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:44.719918    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:47.230039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:47.230001    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:12:50.209410 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:50.209368    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:12:59.926160 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:12:59.926125    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:13:14.855663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:14.855580    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:13:39.066203 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:39.066167    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:13:50.100337 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:50.100304    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:13:50.246111 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:50.246070    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:13:51.486963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:51.486930    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:13:51.567203 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:13:51.567163    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:14:29.009403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:14:29.009364    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:14:39.999834 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:14:39.999778    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:14:48.375431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:14:48.375340    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:14:50.503541 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:14:50.503504    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:15:00.154813 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.154749    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"]
Jul 22 12:15:00.158209 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.158186    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"
Jul 22 12:15:00.160755 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.160736    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:15:00.167740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.167714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"
Jul 22 12:15:00.168943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.168922    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"]
Jul 22 12:15:00.302993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.302972    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"]
Jul 22 12:15:00.305410 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:15:00.305383    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec7b5d0_71ee_451d_b757_051825d57886.slice/crio-75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1 WatchSource:0}: Error finding container 75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1: Status 404 returned error can't find the container with id 75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1
Jul 22 12:15:00.307089 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:00.307073    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:15:01.301598 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:01.301562    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w" event={"ID":"aec7b5d0-71ee-451d-b757-051825d57886","Type":"ContainerStarted","Data":"75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1"}
Jul 22 12:15:03.312776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:03.312734    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w" event={"ID":"aec7b5d0-71ee-451d-b757-051825d57886","Type":"ContainerStarted","Data":"c1ff8cec0b09b2407eb667b95c5b5a8b7351e1a2fee61dc4e4bcc483edceb870"}
Jul 22 12:15:03.330171 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:03.330119    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w" podStartSLOduration=0.977387473 podStartE2EDuration="3.330099686s" totalImagesPullingTime="2.352712213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:15:03.329299186 +0000 UTC m=+823.752730241" watchObservedRunningTime="2026-07-22 12:15:03.330099686 +0000 UTC m=+823.753530743"
Jul 22 12:15:04.318464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:04.318427    2565 generic.go:356] "Generic (PLEG): container finished" podID="aec7b5d0-71ee-451d-b757-051825d57886" containerID="c1ff8cec0b09b2407eb667b95c5b5a8b7351e1a2fee61dc4e4bcc483edceb870" exitCode=0
Jul 22 12:15:04.318874 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:04.318511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w" event={"ID":"aec7b5d0-71ee-451d-b757-051825d57886","Type":"ContainerDied","Data":"c1ff8cec0b09b2407eb667b95c5b5a8b7351e1a2fee61dc4e4bcc483edceb870"}
Jul 22 12:15:05.450152 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:05.450129    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"
Jul 22 12:15:06.328381 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:06.328353    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w"
Jul 22 12:15:06.328551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:06.328353    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745375-cxf8w" event={"ID":"aec7b5d0-71ee-451d-b757-051825d57886","Type":"ContainerDied","Data":"75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1"}
Jul 22 12:15:06.328551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:06.328482    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75b529494cef5f87455503435b8174560aa177bd3e0666b03b17d03b44d9cdd1"
Jul 22 12:15:18.016778 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:18.016735    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:15:18.098364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:15:18.098330    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:06.807368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:06.807330    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:08.040974 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:08.040943    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:33.096123 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:33.096076    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:16:41.104344 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:41.104308    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:16:43.794768 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:43.794734    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:43.894578 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:43.894542    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:43.998152 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:43.998110    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:50.562063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:50.562017    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:50.687609 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:50.687565    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:50.786811 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:50.786755    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:50.917396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:50.917303    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:57.164328 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:57.164291    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:16:57.362924 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:16:57.362889    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:17:13.697428 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:17:13.697389    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:17:23.905938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:17:23.905909    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:17:34.398589 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:17:34.398546    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:17:42.807498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:17:42.807418    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:17:54.101717 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:17:54.101681    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:18:03.699619 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:18:03.699576    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:18:14.894906 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:18:14.894871    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:18:24.816261 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:18:24.816225    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:18:47.233871 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:18:47.233839    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:18:56.406709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:18:56.406670    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:19:20.807905 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:19:20.807826    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:19:30.897378 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:19:30.897344    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:19:47.799889 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:19:47.799850    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:19:56.903709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:19:56.903674    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:20:06.904884 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:20:06.904849    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:20:16.694728 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:20:16.694690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:20:25.908501 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:20:25.908464    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:21:59.638919 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:21:59.638886    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:22:01.468353 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:01.468316    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:22:14.392042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:14.391955    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:22:19.655521 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:19.655485    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:22:26.110736 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:26.110698    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:22:34.701812 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:34.701759    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:22:44.900288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:44.900254    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:22:55.906487 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:55.906450    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:22:59.902011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:22:59.901974    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:23:06.095756 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:06.095721    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:23:16.512038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:16.512003    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:23:26.291365 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:26.291327    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:23:38.413611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:38.413569    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:23:47.102300 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:47.102214    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:23:58.411717 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:23:58.411680    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:24:19.412558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:19.412521    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:24:28.897221 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:28.897182    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:24:30.483213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:30.483171    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:24:42.292615 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:42.292558    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:24:43.982146 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.982111    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:24:43.982494 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.982404    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec7b5d0-71ee-451d-b757-051825d57886" containerName="cleanup"
Jul 22 12:24:43.982494 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.982415    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec7b5d0-71ee-451d-b757-051825d57886" containerName="cleanup"
Jul 22 12:24:43.982494 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.982467    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec7b5d0-71ee-451d-b757-051825d57886" containerName="cleanup"
Jul 22 12:24:43.985198 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.985177    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:43.987436 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.987401    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 12:24:43.987546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.987468    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 12:24:43.988577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.988557    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gf6j9\""
Jul 22 12:24:43.988697 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.988557    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-be372d4c-kserve-self-signed-certs\""
Jul 22 12:24:43.997376 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:43.997357    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:24:44.023992 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.023963    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:24:44.112066 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112033    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.112245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112097    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.112245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5ncj\" (UniqueName: \"kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.112245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112159    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.112245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112185    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.112245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.112204    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.212856 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.212823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.212856 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.212861    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.212876    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213006    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213192 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213354 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213324    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213422 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5ncj\" (UniqueName: \"kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213478 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.213704 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.213681    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.215297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.215269    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.215395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.215362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.222564 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.222539    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5ncj\" (UniqueName: \"kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj\") pod \"e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") " pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.294390 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.294355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:44.427679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.427647    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:24:44.431230 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:24:44.431196    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod072ba648_8967_40b4_9a1b_bc24ea8dc317.slice/crio-c382ff25bcd0145f97f8529bfb88a4323c8ae9ee48e0cae82ca4efb068d83a80 WatchSource:0}: Error finding container c382ff25bcd0145f97f8529bfb88a4323c8ae9ee48e0cae82ca4efb068d83a80: Status 404 returned error can't find the container with id c382ff25bcd0145f97f8529bfb88a4323c8ae9ee48e0cae82ca4efb068d83a80
Jul 22 12:24:44.433451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.433434    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:24:44.713311 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:44.713216    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" event={"ID":"072ba648-8967-40b4-9a1b-bc24ea8dc317","Type":"ContainerStarted","Data":"c382ff25bcd0145f97f8529bfb88a4323c8ae9ee48e0cae82ca4efb068d83a80"}
Jul 22 12:24:45.718692 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:45.718604    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" event={"ID":"072ba648-8967-40b4-9a1b-bc24ea8dc317","Type":"ContainerStarted","Data":"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"}
Jul 22 12:24:45.735480 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:45.735426    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" podStartSLOduration=1.746100079 podStartE2EDuration="2.735412307s" totalImagesPullingTime="989.312228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:24:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:24:45.73514778 +0000 UTC m=+1406.158578835" watchObservedRunningTime="2026-07-22 12:24:45.735412307 +0000 UTC m=+1406.158843361"
Jul 22 12:24:53.807032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:53.806994    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:24:54.294831 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:54.294778    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:54.295022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:54.294849    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:54.307565 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:54.307541    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:24:54.768617 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:24:54.768590    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:25:06.457002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:06.456963    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:25:07.348532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:07.348494    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:25:07.367431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:07.367406    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:25:09.291725 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:09.291690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:25:22.369130 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.369016    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" podUID="072ba648-8967-40b4-9a1b-bc24ea8dc317" containerName="main" containerID="cri-o://9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d" gracePeriod=45
Jul 22 12:25:22.611972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.611943    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:25:22.630646 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630551    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630646 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630592    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630646 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630623    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630940 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630665    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630940 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630717    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630940 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630772    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5ncj\" (UniqueName: \"kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj\") pod \"072ba648-8967-40b4-9a1b-bc24ea8dc317\" (UID: \"072ba648-8967-40b4-9a1b-bc24ea8dc317\") "
Jul 22 12:25:22.630940 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630811    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home" (OuterVolumeSpecName: "home") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:25:22.630940 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.630918    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache" (OuterVolumeSpecName: "model-cache") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:25:22.631196 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.631046    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.631196 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.631065    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.631409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.631355    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:25:22.633440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.633405    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj" (OuterVolumeSpecName: "kube-api-access-r5ncj") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "kube-api-access-r5ncj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:25:22.633440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.633408    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:25:22.633440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.633410    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm" (OuterVolumeSpecName: "dshm") pod "072ba648-8967-40b4-9a1b-bc24ea8dc317" (UID: "072ba648-8967-40b4-9a1b-bc24ea8dc317"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:25:22.732241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.732209    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5ncj\" (UniqueName: \"kubernetes.io/projected/072ba648-8967-40b4-9a1b-bc24ea8dc317-kube-api-access-r5ncj\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.732241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.732239    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072ba648-8967-40b4-9a1b-bc24ea8dc317-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.732241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.732250    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.732241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.732259    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072ba648-8967-40b4-9a1b-bc24ea8dc317-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:25:22.877235 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.877197    2565 generic.go:356] "Generic (PLEG): container finished" podID="072ba648-8967-40b4-9a1b-bc24ea8dc317" containerID="9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d" exitCode=0
Jul 22 12:25:22.877235 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.877236    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"
Jul 22 12:25:22.877419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.877278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" event={"ID":"072ba648-8967-40b4-9a1b-bc24ea8dc317","Type":"ContainerDied","Data":"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"}
Jul 22 12:25:22.877419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.877322    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg" event={"ID":"072ba648-8967-40b4-9a1b-bc24ea8dc317","Type":"ContainerDied","Data":"c382ff25bcd0145f97f8529bfb88a4323c8ae9ee48e0cae82ca4efb068d83a80"}
Jul 22 12:25:22.877419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.877340    2565 scope.go:117] "RemoveContainer" containerID="9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"
Jul 22 12:25:22.889024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.889007    2565 scope.go:117] "RemoveContainer" containerID="9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"
Jul 22 12:25:22.889284 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:25:22.889266    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d\": container with ID starting with 9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d not found: ID does not exist" containerID="9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"
Jul 22 12:25:22.889330 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.889294    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d"} err="failed to get container status \"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d\": rpc error: code = NotFound desc = could not find container \"9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d\": container with ID starting with 9f5c3163be3e9d7468e774fc74124be7706f0baa25f85b68bb4b272629d8422d not found: ID does not exist"
Jul 22 12:25:22.902077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.902050    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:25:22.906517 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:22.906496    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-be372d4c-kserve-657b45659f-mz9lg"]
Jul 22 12:25:23.398215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:23.398181    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:25:24.133611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:24.133578    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="072ba648-8967-40b4-9a1b-bc24ea8dc317" path="/var/lib/kubelet/pods/072ba648-8967-40b4-9a1b-bc24ea8dc317/volumes"
Jul 22 12:25:32.203899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:32.203857    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:25:45.289150 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:45.289112    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:25:56.504073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:25:56.504039    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:06.201109 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:06.201072    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:14.198266 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:14.198219    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:24.003615 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:24.003574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:33.800548 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:33.800508    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:47.245371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:47.245297    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:26:56.302866 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:26:56.302831    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:27:05.702603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:05.702545    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:27:18.402840 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:18.402799    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:27:21.921026 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:21.920989    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:27:27.399053 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:27.399019    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:27:34.300243 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:34.300210    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:27:49.209573 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:27:49.209535    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:00.508383 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:00.508344    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:09.703655 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:09.703619    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:12.173063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:12.173024    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:23.679112 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:23.679068    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:24.202950 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:24.202914    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:33.607442 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:33.607410    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:33.657129 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:33.657092    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:36.699096 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:36.699054    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:36.797463 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:36.797430    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:37.142412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:37.142378    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:43.210224 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.210188    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:43.255517 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.255484    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:43.536496 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.536459    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:43.637188 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.637152    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:43.873614 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.873530    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:28:43.873937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.873922    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="072ba648-8967-40b4-9a1b-bc24ea8dc317" containerName="main"
Jul 22 12:28:43.873989 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.873939    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="072ba648-8967-40b4-9a1b-bc24ea8dc317" containerName="main"
Jul 22 12:28:43.874027 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.873994    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="072ba648-8967-40b4-9a1b-bc24ea8dc317" containerName="main"
Jul 22 12:28:43.876428 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.876411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.878590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.878572    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-20d1eb60-openshift-default-dockercfg-nttzz\""
Jul 22 12:28:43.900582 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.900554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:28:43.902277 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.902255    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:43.980621 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980621 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980623    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwhhn\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:43.980987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:43.980898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.081757 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081727    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwhhn\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.081757 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081828    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081874    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.081954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082381 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.082259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082381 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.082363    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082484 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.082461    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082601 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.082579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.082643 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.082604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.084235 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.084215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.084343 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.084327    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.085529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.085508    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:44.089668 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.089642    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.089668 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.089659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwhhn\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn\") pod \"e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") " pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.186365 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.186295    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:44.322828 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.322802    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:28:44.325007 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:28:44.324979    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429a18bb_1d3b_4ebf_bac8_d55bf5db4cd7.slice/crio-e05cb15c2be8b28078757ade69bc3b571979ad10aeed211300873efeec40e0a2 WatchSource:0}: Error finding container e05cb15c2be8b28078757ade69bc3b571979ad10aeed211300873efeec40e0a2: Status 404 returned error can't find the container with id e05cb15c2be8b28078757ade69bc3b571979ad10aeed211300873efeec40e0a2
Jul 22 12:28:44.327208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.327175    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:28:44.327283 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.327247    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:28:44.327324 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.327298    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:28:44.736433 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.736395    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerStarted","Data":"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"}
Jul 22 12:28:44.736732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.736454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerStarted","Data":"e05cb15c2be8b28078757ade69bc3b571979ad10aeed211300873efeec40e0a2"}
Jul 22 12:28:44.755771 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:44.755726    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podStartSLOduration=1.755710297 podStartE2EDuration="1.755710297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:28:44.75410282 +0000 UTC m=+1645.177533875" watchObservedRunningTime="2026-07-22 12:28:44.755710297 +0000 UTC m=+1645.179141351"
Jul 22 12:28:45.186893 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:45.186853    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:45.191682 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:45.191659    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:45.740930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:45.740895    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:45.741995 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:45.741978    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:28:46.104355 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.104323    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:46.189325 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.189290    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:46.271410 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.271363    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:46.380004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.379916    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:46.831254 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.831217    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:46.931332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:46.931295    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:28:51.392122 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:51.392082    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:28:59.799118 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.799085    2565 generic.go:356] "Generic (PLEG): container finished" podID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerID="73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731" exitCode=0
Jul 22 12:28:59.799590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.799157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerDied","Data":"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"}
Jul 22 12:28:59.799590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.799502    2565 scope.go:117] "RemoveContainer" containerID="73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"
Jul 22 12:28:59.800513 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.800482    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:28:59.800624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.800564    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:28:59.800624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:28:59.800615    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:29:00.741122 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:00.741077    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:00.804996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:00.804963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerStarted","Data":"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"}
Jul 22 12:29:01.186888 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:01.186852    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:01.188135 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:01.188101    2565 patch_prober.go:28] interesting pod/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 22 12:29:01.188259 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:01.188166    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 22 12:29:02.187382 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:02.187348    2565 patch_prober.go:28] interesting pod/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 22 12:29:02.187748 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:02.187405    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 22 12:29:03.187340 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:03.187297    2565 patch_prober.go:28] interesting pod/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 22 12:29:03.187517 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:03.187356    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 22 12:29:04.187525 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:04.187481    2565 patch_prober.go:28] interesting pod/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Jul 22 12:29:04.188022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:04.187547    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Jul 22 12:29:05.190559 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:05.190528    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:05.191021 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:05.190817    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:05.191397 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:05.191381    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:11.849270 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.849233    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"]
Jul 22 12:29:11.852485 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.852468    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:11.854763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.854734    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:29:11.854763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.854753    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:29:11.859916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.859887    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"]
Jul 22 12:29:11.931413 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:11.931369    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9\" (UID: \"096fde6c-585c-4bb8-91ff-9b05d22135a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:12.032777 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.032735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9\" (UID: \"096fde6c-585c-4bb8-91ff-9b05d22135a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:12.033404 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.033379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9\" (UID: \"096fde6c-585c-4bb8-91ff-9b05d22135a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:12.163818 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.163710    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:12.496680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.496480    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"]
Jul 22 12:29:12.499210 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:29:12.499183    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096fde6c_585c_4bb8_91ff_9b05d22135a9.slice/crio-3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388 WatchSource:0}: Error finding container 3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388: Status 404 returned error can't find the container with id 3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388
Jul 22 12:29:12.863748 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.863712    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9" event={"ID":"096fde6c-585c-4bb8-91ff-9b05d22135a9","Type":"ContainerStarted","Data":"726046d5ab4a8b5bd8a1ce92da6c88b30a7599c55ff5e49d6360d39091eceb8e"}
Jul 22 12:29:12.864160 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.863767    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9" event={"ID":"096fde6c-585c-4bb8-91ff-9b05d22135a9","Type":"ContainerStarted","Data":"3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388"}
Jul 22 12:29:12.879191 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:12.879147    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9" podStartSLOduration=1.879132553 podStartE2EDuration="1.879132553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:29:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:29:12.87903934 +0000 UTC m=+1673.302470395" watchObservedRunningTime="2026-07-22 12:29:12.879132553 +0000 UTC m=+1673.302563607"
Jul 22 12:29:13.869456 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:13.869422    2565 generic.go:356] "Generic (PLEG): container finished" podID="096fde6c-585c-4bb8-91ff-9b05d22135a9" containerID="726046d5ab4a8b5bd8a1ce92da6c88b30a7599c55ff5e49d6360d39091eceb8e" exitCode=0
Jul 22 12:29:13.869456 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:13.869458    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9" event={"ID":"096fde6c-585c-4bb8-91ff-9b05d22135a9","Type":"ContainerDied","Data":"726046d5ab4a8b5bd8a1ce92da6c88b30a7599c55ff5e49d6360d39091eceb8e"}
Jul 22 12:29:15.003415 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.003387    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:15.056427 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.056396    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca\") pod \"096fde6c-585c-4bb8-91ff-9b05d22135a9\" (UID: \"096fde6c-585c-4bb8-91ff-9b05d22135a9\") "
Jul 22 12:29:15.056743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.056720    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "096fde6c-585c-4bb8-91ff-9b05d22135a9" (UID: "096fde6c-585c-4bb8-91ff-9b05d22135a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:29:15.157979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.157889    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/096fde6c-585c-4bb8-91ff-9b05d22135a9-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:15.879199 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.879164    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9" event={"ID":"096fde6c-585c-4bb8-91ff-9b05d22135a9","Type":"ContainerDied","Data":"3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388"}
Jul 22 12:29:15.879367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.879212    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bd9eb033de336fd428e528f4d76d65587a87bcfde677d0eb0fab0438d522388"
Jul 22 12:29:15.879367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:15.879173    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"
Jul 22 12:29:22.335897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:22.335859    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:25.294302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:25.294262    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:29:26.939746 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:26.939704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:26.943776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:26.943746    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"]
Jul 22 12:29:26.947383 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:26.947362    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-20d1eb60-98kq9"]
Jul 22 12:29:27.093575 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.093544    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:29:27.093772 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.093751    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy" containerID="cri-o://a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d" gracePeriod=30
Jul 22 12:29:27.112460 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.112436    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:27.404375 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.404331    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:27.697558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.697479    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:27.793761 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.793727    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:27.909658 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:27.909620    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:28.133272 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:28.133239    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096fde6c-585c-4bb8-91ff-9b05d22135a9" path="/var/lib/kubelet/pods/096fde6c-585c-4bb8-91ff-9b05d22135a9/volumes"
Jul 22 12:29:28.277211 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:28.277174    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:28.357946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:28.357911    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:30.476529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:30.476493    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:30.594498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:30.594458    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:30.847428 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:30.847392    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:31.094372 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:31.094337    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:31.195631 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:31.195546    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:29:32.343528 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.343506    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:32.400496 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400417    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400496 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400458    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400496 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400718 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400544    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400718 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400592    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400718 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400622    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400718 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400718 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400692    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwhhn\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400724    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy\") pod \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\" (UID: \"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7\") "
Jul 22 12:29:32.400927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400887    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:29:32.400927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400895    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:29:32.401049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.400997    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.401049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.401014    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.401256 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.401225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:29:32.401382 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.401250    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:29:32.401471 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.401450    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:29:32.403002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.402977    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:29:32.403110 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.403011    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:29:32.403168 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.403142    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn" (OuterVolumeSpecName: "kube-api-access-fwhhn") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "kube-api-access-fwhhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:29:32.403228 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.403178    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" (UID: "429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:29:32.502117 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502081    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwhhn\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-kube-api-access-fwhhn\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502117 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502112    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502117 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502120    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502117 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502128    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502136    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502144    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.502370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.502152    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:32.955187 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.955145    2565 generic.go:356] "Generic (PLEG): container finished" podID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerID="a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d" exitCode=0
Jul 22 12:29:32.955331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.955198    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"
Jul 22 12:29:32.955331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.955202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerDied","Data":"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"}
Jul 22 12:29:32.955331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.955255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj" event={"ID":"429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7","Type":"ContainerDied","Data":"e05cb15c2be8b28078757ade69bc3b571979ad10aeed211300873efeec40e0a2"}
Jul 22 12:29:32.955331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.955285    2565 scope.go:117] "RemoveContainer" containerID="a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"
Jul 22 12:29:32.964420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.964400    2565 scope.go:117] "RemoveContainer" containerID="73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"
Jul 22 12:29:32.972108 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.972091    2565 scope.go:117] "RemoveContainer" containerID="a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"
Jul 22 12:29:32.972348 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:29:32.972327    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d\": container with ID starting with a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d not found: ID does not exist" containerID="a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"
Jul 22 12:29:32.972391 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.972356    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d"} err="failed to get container status \"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d\": rpc error: code = NotFound desc = could not find container \"a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d\": container with ID starting with a82dcb31b336b25b8f9307b23f7b2859fd798af7b5c41e2c0a7465cca55c269d not found: ID does not exist"
Jul 22 12:29:32.972391 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.972375    2565 scope.go:117] "RemoveContainer" containerID="73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"
Jul 22 12:29:32.972612 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:29:32.972596    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731\": container with ID starting with 73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731 not found: ID does not exist" containerID="73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"
Jul 22 12:29:32.972662 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.972616    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731"} err="failed to get container status \"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731\": rpc error: code = NotFound desc = could not find container \"73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731\": container with ID starting with 73ba7a671bdc22597d6cb2dc643bee611c2e3426ebaa0395c798cfaec1155731 not found: ID does not exist"
Jul 22 12:29:32.978073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.978051    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:29:32.986361 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:32.984358    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-20d1eb60-openshift-default-c86775b95-mjsjj"]
Jul 22 12:29:34.133854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:34.133821    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" path="/var/lib/kubelet/pods/429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7/volumes"
Jul 22 12:29:36.301449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:36.301415    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:29:55.572408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572367    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"]
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572851    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572870    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572884    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572892    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572901    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096fde6c-585c-4bb8-91ff-9b05d22135a9" containerName="revoke-keys"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572909    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="096fde6c-585c-4bb8-91ff-9b05d22135a9" containerName="revoke-keys"
Jul 22 12:29:55.573004 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.572999    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573341 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.573012    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="429a18bb-1d3b-4ebf-bac8-d55bf5db4cd7" containerName="istio-proxy"
Jul 22 12:29:55.573341 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.573022    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="096fde6c-585c-4bb8-91ff-9b05d22135a9" containerName="revoke-keys"
Jul 22 12:29:55.576009 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.575984    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:55.578030 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.578007    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:29:55.578153 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.578101    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:29:55.584744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.584717    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"]
Jul 22 12:29:55.706623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.706588    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm\" (UID: \"42207848-dd2a-4bb3-a4d3-c22834f6281e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:55.807551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.807513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm\" (UID: \"42207848-dd2a-4bb3-a4d3-c22834f6281e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:55.808237 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.808203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm\" (UID: \"42207848-dd2a-4bb3-a4d3-c22834f6281e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:55.887244 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:55.887166    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:56.018846 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:56.018814    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"]
Jul 22 12:29:56.021658 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:29:56.021634    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42207848_dd2a_4bb3_a4d3_c22834f6281e.slice/crio-b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866 WatchSource:0}: Error finding container b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866: Status 404 returned error can't find the container with id b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866
Jul 22 12:29:56.054023 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:56.054001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm" event={"ID":"42207848-dd2a-4bb3-a4d3-c22834f6281e","Type":"ContainerStarted","Data":"b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866"}
Jul 22 12:29:57.059595 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:57.059560    2565 generic.go:356] "Generic (PLEG): container finished" podID="42207848-dd2a-4bb3-a4d3-c22834f6281e" containerID="2af1069450b83fcc0ded78f20d336527478869476559a81ce9c3309179be8634" exitCode=0
Jul 22 12:29:57.060003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:57.059641    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm" event={"ID":"42207848-dd2a-4bb3-a4d3-c22834f6281e","Type":"ContainerDied","Data":"2af1069450b83fcc0ded78f20d336527478869476559a81ce9c3309179be8634"}
Jul 22 12:29:58.192377 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:58.192352    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:58.329635 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:58.329550    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca\") pod \"42207848-dd2a-4bb3-a4d3-c22834f6281e\" (UID: \"42207848-dd2a-4bb3-a4d3-c22834f6281e\") "
Jul 22 12:29:58.329955 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:58.329924    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42207848-dd2a-4bb3-a4d3-c22834f6281e" (UID: "42207848-dd2a-4bb3-a4d3-c22834f6281e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:29:58.431067 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:58.431033    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42207848-dd2a-4bb3-a4d3-c22834f6281e-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:29:59.068716 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:59.068686    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"
Jul 22 12:29:59.068944 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:59.068717    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm" event={"ID":"42207848-dd2a-4bb3-a4d3-c22834f6281e","Type":"ContainerDied","Data":"b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866"}
Jul 22 12:29:59.068944 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:29:59.068760    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b854a766c81c90cde882a4d99c35dbe0512c09a1634be5af4a6b3258488f9866"
Jul 22 12:30:00.139596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.139564    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"]
Jul 22 12:30:00.140132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.139855    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42207848-dd2a-4bb3-a4d3-c22834f6281e" containerName="revoke-keys"
Jul 22 12:30:00.140132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.139867    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="42207848-dd2a-4bb3-a4d3-c22834f6281e" containerName="revoke-keys"
Jul 22 12:30:00.140132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.139939    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="42207848-dd2a-4bb3-a4d3-c22834f6281e" containerName="revoke-keys"
Jul 22 12:30:00.142656 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.142635    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"]
Jul 22 12:30:00.142805 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.142719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"
Jul 22 12:30:00.144934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.144916    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:30:00.146366 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.146346    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"
Jul 22 12:30:00.150105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.150084    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"]
Jul 22 12:30:00.154289 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.154266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"]
Jul 22 12:30:00.155921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.155901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"
Jul 22 12:30:00.163842 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.163821    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"
Jul 22 12:30:00.304272 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.304243    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"]
Jul 22 12:30:00.305871 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:30:00.305845    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fa99e8_1019_4242_b7a0_fa6efd69afae.slice/crio-1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133 WatchSource:0}: Error finding container 1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133: Status 404 returned error can't find the container with id 1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133
Jul 22 12:30:00.324602 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:00.324562    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"]
Jul 22 12:30:01.080960 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.080922    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82" event={"ID":"142410e0-9637-4434-8ae8-abb07a63c09b","Type":"ContainerStarted","Data":"d4f5e51e86c36158b390a011485c12e9b720b8d23432e2c0b80de8d22011660d"}
Jul 22 12:30:01.081168 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.081019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82" event={"ID":"142410e0-9637-4434-8ae8-abb07a63c09b","Type":"ContainerStarted","Data":"cd2c6a9e0d41a910c89e36932956aa922ad1b67e91d9ba7566044dd9cc8a5d8c"}
Jul 22 12:30:01.082348 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.082324    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9" event={"ID":"39fa99e8-1019-4242-b7a0-fa6efd69afae","Type":"ContainerStarted","Data":"8b625c647f90215ed41dd921a3b55be4814ef52d068c09fde2f06f722bbe69e5"}
Jul 22 12:30:01.082447 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.082367    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9" event={"ID":"39fa99e8-1019-4242-b7a0-fa6efd69afae","Type":"ContainerStarted","Data":"1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133"}
Jul 22 12:30:01.097271 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.097220    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82" podStartSLOduration=1.097206618 podStartE2EDuration="1.097206618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:01.095808912 +0000 UTC m=+1721.519239962" watchObservedRunningTime="2026-07-22 12:30:01.097206618 +0000 UTC m=+1721.520637651"
Jul 22 12:30:01.112591 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:01.112541    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9" podStartSLOduration=1.11252585 podStartE2EDuration="1.11252585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:01.111391466 +0000 UTC m=+1721.534822522" watchObservedRunningTime="2026-07-22 12:30:01.11252585 +0000 UTC m=+1721.535956945"
Jul 22 12:30:02.088702 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:02.088669    2565 generic.go:356] "Generic (PLEG): container finished" podID="142410e0-9637-4434-8ae8-abb07a63c09b" containerID="d4f5e51e86c36158b390a011485c12e9b720b8d23432e2c0b80de8d22011660d" exitCode=0
Jul 22 12:30:02.089145 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:02.088759    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82" event={"ID":"142410e0-9637-4434-8ae8-abb07a63c09b","Type":"ContainerDied","Data":"d4f5e51e86c36158b390a011485c12e9b720b8d23432e2c0b80de8d22011660d"}
Jul 22 12:30:02.090130 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:02.090107    2565 generic.go:356] "Generic (PLEG): container finished" podID="39fa99e8-1019-4242-b7a0-fa6efd69afae" containerID="8b625c647f90215ed41dd921a3b55be4814ef52d068c09fde2f06f722bbe69e5" exitCode=0
Jul 22 12:30:02.090195 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:02.090145    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9" event={"ID":"39fa99e8-1019-4242-b7a0-fa6efd69afae","Type":"ContainerDied","Data":"8b625c647f90215ed41dd921a3b55be4814ef52d068c09fde2f06f722bbe69e5"}
Jul 22 12:30:03.220685 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:03.220661    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"
Jul 22 12:30:03.253821 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:03.253780    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"
Jul 22 12:30:04.101104 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.101063    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82"
Jul 22 12:30:04.101294 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.101068    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745390-kdt82" event={"ID":"142410e0-9637-4434-8ae8-abb07a63c09b","Type":"ContainerDied","Data":"cd2c6a9e0d41a910c89e36932956aa922ad1b67e91d9ba7566044dd9cc8a5d8c"}
Jul 22 12:30:04.101294 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.101184    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd2c6a9e0d41a910c89e36932956aa922ad1b67e91d9ba7566044dd9cc8a5d8c"
Jul 22 12:30:04.102213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.102188    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"
Jul 22 12:30:04.102320 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.102191    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9" event={"ID":"39fa99e8-1019-4242-b7a0-fa6efd69afae","Type":"ContainerDied","Data":"1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133"}
Jul 22 12:30:04.102363 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:04.102328    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1801ca7322ab173a8aef60cfff20895d0d2809dc17f673c2bc762cf928df8133"
Jul 22 12:30:06.225961 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:06.225906    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:06.236850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:06.236818    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"]
Jul 22 12:30:06.240038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:06.240012    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-0efd59a1-29745390-whdb9"]
Jul 22 12:30:08.133162 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:08.133128    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39fa99e8-1019-4242-b7a0-fa6efd69afae" path="/var/lib/kubelet/pods/39fa99e8-1019-4242-b7a0-fa6efd69afae/volumes"
Jul 22 12:30:09.213239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:09.213205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:30:10.657457 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:10.657425    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"]
Jul 22 12:30:10.660216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:10.660189    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0efd59a1-dm6jm"]
Jul 22 12:30:10.660802 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:10.660762    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:10.825281 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:10.825246    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:11.066758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.066709    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:11.321130 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.321027    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:11.621999 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.621920    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:30:11.622252 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622239    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39fa99e8-1019-4242-b7a0-fa6efd69afae" containerName="cleanup"
Jul 22 12:30:11.622292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622254    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="39fa99e8-1019-4242-b7a0-fa6efd69afae" containerName="cleanup"
Jul 22 12:30:11.622292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622271    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142410e0-9637-4434-8ae8-abb07a63c09b" containerName="cleanup"
Jul 22 12:30:11.622292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622277    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="142410e0-9637-4434-8ae8-abb07a63c09b" containerName="cleanup"
Jul 22 12:30:11.622383 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622329    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="142410e0-9637-4434-8ae8-abb07a63c09b" containerName="cleanup"
Jul 22 12:30:11.622383 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.622336    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="39fa99e8-1019-4242-b7a0-fa6efd69afae" containerName="cleanup"
Jul 22 12:30:11.625268 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.625250    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.627420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.627401    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cf37a858-openshift-default-dockercfg-hzvtn\""
Jul 22 12:30:11.634409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634421    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634486    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634507    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x4t7\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.634723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.634663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.636154 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.636129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:30:11.642512 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.642485    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:11.735808 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.735754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.735832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.735855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.735887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x4t7\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.735940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736096    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736202    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736212    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736356 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736905 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736487    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736905 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.736905 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.736898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.738205 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.738184    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.738761 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.738743    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.744487 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.744463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.744679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.744657    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x4t7\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7\") pod \"e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") " pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:11.755193 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.755159    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:11.936655 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:11.936558    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:12.076358 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.076332    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:30:12.078159 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:30:12.078132    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bcbe46_2453_4e6b_8444_70f9e686e971.slice/crio-f3940ff467808cad309e7d85ad947957b4dd4d770331546b31e71310aa853958 WatchSource:0}: Error finding container f3940ff467808cad309e7d85ad947957b4dd4d770331546b31e71310aa853958: Status 404 returned error can't find the container with id f3940ff467808cad309e7d85ad947957b4dd4d770331546b31e71310aa853958
Jul 22 12:30:12.080213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.080179    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:12.080309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.080260    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:12.080369 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.080329    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:12.136719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.136695    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42207848-dd2a-4bb3-a4d3-c22834f6281e" path="/var/lib/kubelet/pods/42207848-dd2a-4bb3-a4d3-c22834f6281e/volumes"
Jul 22 12:30:12.138717 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:12.138695    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerStarted","Data":"f3940ff467808cad309e7d85ad947957b4dd4d770331546b31e71310aa853958"}
Jul 22 12:30:13.146186 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:13.146149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerStarted","Data":"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"}
Jul 22 12:30:13.167177 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:13.167122    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podStartSLOduration=2.167102663 podStartE2EDuration="2.167102663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:13.165079549 +0000 UTC m=+1733.588510605" watchObservedRunningTime="2026-07-22 12:30:13.167102663 +0000 UTC m=+1733.590533719"
Jul 22 12:30:13.937568 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:13.937524    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:13.942547 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:13.942515    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:14.150301 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:14.150267    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:14.151396 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:14.151379    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:14.544981 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:14.544942    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:14.955399 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:14.955304    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:15.055744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:15.055712    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:15.222817 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:15.222708    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:15.323513 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:15.323475    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:15.427542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:15.427507    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:20.504949 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:20.504854    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:30:29.151831 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.151803    2565 patch_prober.go:28] interesting pod/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:30:29.152165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.151855    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:30:29.217489 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.217456    2565 generic.go:356] "Generic (PLEG): container finished" podID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerID="deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb" exitCode=0
Jul 22 12:30:29.217648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.217527    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerDied","Data":"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"}
Jul 22 12:30:29.217910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.217896    2565 scope.go:117] "RemoveContainer" containerID="deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"
Jul 22 12:30:29.218848 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.218822    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:29.218916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.218884    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:29.218956 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:29.218917    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:30.224020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:30.223977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerStarted","Data":"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"}
Jul 22 12:30:30.937610 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:30.937566    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:30.937839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:30.937702    2565 patch_prober.go:28] interesting pod/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:30:30.937839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:30.937763    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:30:31.938010 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:31.937964    2565 patch_prober.go:28] interesting pod/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:30:31.938505 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:31.938049    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:30:32.937492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:32.937441    2565 patch_prober.go:28] interesting pod/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:30:32.937711 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:32.937531    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:30:33.943257 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:33.943204    2565 patch_prober.go:28] interesting pod/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:30:33.943756 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:33.943281    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:30:34.941027 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:34.940994    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:34.941372 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:34.941353    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:34.941949 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:34.941929    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:30:40.412812 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.412766    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"]
Jul 22 12:30:40.416213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.416197    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:40.418684 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.418661    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:30:40.418829 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.418694    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:30:40.424877 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.424848    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"]
Jul 22 12:30:40.483776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.483734    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm\" (UID: \"28804d7a-35f7-4bda-a4dd-37731d6b31d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:40.584453 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.584414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm\" (UID: \"28804d7a-35f7-4bda-a4dd-37731d6b31d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:40.585052 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.585033    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm\" (UID: \"28804d7a-35f7-4bda-a4dd-37731d6b31d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:40.728173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.728080    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:40.863378 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:40.863353    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"]
Jul 22 12:30:40.865987 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:30:40.865946    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28804d7a_35f7_4bda_a4dd_37731d6b31d3.slice/crio-7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef WatchSource:0}: Error finding container 7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef: Status 404 returned error can't find the container with id 7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef
Jul 22 12:30:41.278354 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:41.278316    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm" event={"ID":"28804d7a-35f7-4bda-a4dd-37731d6b31d3","Type":"ContainerStarted","Data":"ea7487c2e347f76010651edf7ec114732330b76660cb8e860e0e6859367ddecf"}
Jul 22 12:30:41.278558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:41.278378    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm" event={"ID":"28804d7a-35f7-4bda-a4dd-37731d6b31d3","Type":"ContainerStarted","Data":"7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef"}
Jul 22 12:30:41.296378 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:41.296326    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm" podStartSLOduration=1.296309388 podStartE2EDuration="1.296309388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:41.294914816 +0000 UTC m=+1761.718345872" watchObservedRunningTime="2026-07-22 12:30:41.296309388 +0000 UTC m=+1761.719740443"
Jul 22 12:30:42.289701 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:42.289663    2565 generic.go:356] "Generic (PLEG): container finished" podID="28804d7a-35f7-4bda-a4dd-37731d6b31d3" containerID="ea7487c2e347f76010651edf7ec114732330b76660cb8e860e0e6859367ddecf" exitCode=0
Jul 22 12:30:42.290106 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:42.289709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm" event={"ID":"28804d7a-35f7-4bda-a4dd-37731d6b31d3","Type":"ContainerDied","Data":"ea7487c2e347f76010651edf7ec114732330b76660cb8e860e0e6859367ddecf"}
Jul 22 12:30:43.433648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:43.433623    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:43.507464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:43.507432    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca\") pod \"28804d7a-35f7-4bda-a4dd-37731d6b31d3\" (UID: \"28804d7a-35f7-4bda-a4dd-37731d6b31d3\") "
Jul 22 12:30:43.507825 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:43.507781    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28804d7a-35f7-4bda-a4dd-37731d6b31d3" (UID: "28804d7a-35f7-4bda-a4dd-37731d6b31d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:30:43.608760 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:43.608655    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28804d7a-35f7-4bda-a4dd-37731d6b31d3-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:30:44.299204 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:44.299178    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"
Jul 22 12:30:44.299385 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:44.299173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm" event={"ID":"28804d7a-35f7-4bda-a4dd-37731d6b31d3","Type":"ContainerDied","Data":"7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef"}
Jul 22 12:30:44.299385 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:44.299308    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e1e92abac913e6437b3ef86b7daf6a8a0d8297f488b84f411a079b1862c73ef"
Jul 22 12:30:50.996848 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:50.996811    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:53.914696 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:53.914662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:30:55.494986 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:55.494948    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:55.506205 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:55.506168    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"]
Jul 22 12:30:55.506420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:55.506396    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf37a858-rc5sm"]
Jul 22 12:30:56.133364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:56.133329    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28804d7a-35f7-4bda-a4dd-37731d6b31d3" path="/var/lib/kubelet/pods/28804d7a-35f7-4bda-a4dd-37731d6b31d3/volumes"
Jul 22 12:30:56.729697 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:56.729657    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:30:56.730140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:56.729921    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy" containerID="cri-o://73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c" gracePeriod=30
Jul 22 12:30:56.752863 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:56.752827    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:56.971067 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:56.971036    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:57.215138 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.215096    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:57.527344 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.527309    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:30:57.527621 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.527608    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28804d7a-35f7-4bda-a4dd-37731d6b31d3" containerName="revoke-keys"
Jul 22 12:30:57.527667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.527623    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="28804d7a-35f7-4bda-a4dd-37731d6b31d3" containerName="revoke-keys"
Jul 22 12:30:57.527701 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.527682    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="28804d7a-35f7-4bda-a4dd-37731d6b31d3" containerName="revoke-keys"
Jul 22 12:30:57.530431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.530406    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.537073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.537047    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a7ea4ba9-openshift-default-dockercfg-rdrj8\""
Jul 22 12:30:57.548411 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.548384    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:30:57.629527 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629486    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629720 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629547    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629720 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629582    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629720 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.629946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.629900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2rnf\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.657143 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.657112    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:57.730508 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2rnf\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.730987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.730937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.731440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.731168    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.731626 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.731596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.731833 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.731810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.732046 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.732024    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.732150 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.732114    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.734059 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.734039    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.734284 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.734267    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.739801 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.739772    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.739923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.739829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2rnf\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf\") pod \"e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") " pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.840123 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.840036    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:57.940210 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.940173    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:57.981311 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.981283    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:30:57.983685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:30:57.983651    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad56dccc_dd34_4508_b32f_456033007eca.slice/crio-f910b75b8f0fa58e1952f333f6390b57aa4bb7ba43bea9b9bf5f70ae374cab6e WatchSource:0}: Error finding container f910b75b8f0fa58e1952f333f6390b57aa4bb7ba43bea9b9bf5f70ae374cab6e: Status 404 returned error can't find the container with id f910b75b8f0fa58e1952f333f6390b57aa4bb7ba43bea9b9bf5f70ae374cab6e
Jul 22 12:30:57.985909 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.985880    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:57.985987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.985948    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:57.985987 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:57.985976    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:30:58.361508 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:58.361473    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerStarted","Data":"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"}
Jul 22 12:30:58.361681 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:58.361526    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerStarted","Data":"f910b75b8f0fa58e1952f333f6390b57aa4bb7ba43bea9b9bf5f70ae374cab6e"}
Jul 22 12:30:58.382412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:58.382365    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podStartSLOduration=1.382348056 podStartE2EDuration="1.382348056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:30:58.380945907 +0000 UTC m=+1778.804376996" watchObservedRunningTime="2026-07-22 12:30:58.382348056 +0000 UTC m=+1778.805779180"
Jul 22 12:30:58.840202 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:58.840163    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:59.756062 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:59.756027    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:59.845531 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:59.845493    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:30:59.874572 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:59.874539    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:30:59.983498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:30:59.983458    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:00.315306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:00.315270    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:00.372078 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:00.372036    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:00.373098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:00.373075    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:00.541509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:00.541467    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:00.641977 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:00.641887    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:01.981038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:01.981013    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:31:02.066889 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.066855    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.066913    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.066934    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.066978    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.066998    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067038    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067065    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067094    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.067408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067305    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:02.067408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067319    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:02.067567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067499    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:02.067689 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067656    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:31:02.067844 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.067777    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:02.069225 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.069205    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:31:02.069524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.069500    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:31:02.069612 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.069519    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:02.167957 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.167922    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x4t7\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7\") pod \"f6bcbe46-2453-4e6b-8444-70f9e686e971\" (UID: \"f6bcbe46-2453-4e6b-8444-70f9e686e971\") "
Jul 22 12:31:02.168238 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168225    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168240    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168250    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bcbe46-2453-4e6b-8444-70f9e686e971-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168258    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168266    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168275    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168295 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168283    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.168480 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.168299    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bcbe46-2453-4e6b-8444-70f9e686e971-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.169995 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.169969    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7" (OuterVolumeSpecName: "kube-api-access-4x4t7") pod "f6bcbe46-2453-4e6b-8444-70f9e686e971" (UID: "f6bcbe46-2453-4e6b-8444-70f9e686e971"). InnerVolumeSpecName "kube-api-access-4x4t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:31:02.269438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.269392    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x4t7\" (UniqueName: \"kubernetes.io/projected/f6bcbe46-2453-4e6b-8444-70f9e686e971-kube-api-access-4x4t7\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:02.383686 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.383598    2565 generic.go:356] "Generic (PLEG): container finished" podID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerID="73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c" exitCode=0
Jul 22 12:31:02.383686 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.383653    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"
Jul 22 12:31:02.383945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.383652    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerDied","Data":"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"}
Jul 22 12:31:02.383945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.383773    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz" event={"ID":"f6bcbe46-2453-4e6b-8444-70f9e686e971","Type":"ContainerDied","Data":"f3940ff467808cad309e7d85ad947957b4dd4d770331546b31e71310aa853958"}
Jul 22 12:31:02.383945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.383823    2565 scope.go:117] "RemoveContainer" containerID="73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"
Jul 22 12:31:02.392434 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.392415    2565 scope.go:117] "RemoveContainer" containerID="deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"
Jul 22 12:31:02.400836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.400814    2565 scope.go:117] "RemoveContainer" containerID="73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"
Jul 22 12:31:02.401122 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:31:02.401099    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c\": container with ID starting with 73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c not found: ID does not exist" containerID="73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"
Jul 22 12:31:02.401200 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.401130    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c"} err="failed to get container status \"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c\": rpc error: code = NotFound desc = could not find container \"73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c\": container with ID starting with 73d183fab2bec4187a1504ac6c0972e4633d6fa2bc797799231b56e0a65eb85c not found: ID does not exist"
Jul 22 12:31:02.401200 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.401146    2565 scope.go:117] "RemoveContainer" containerID="deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"
Jul 22 12:31:02.401387 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:31:02.401371    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb\": container with ID starting with deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb not found: ID does not exist" containerID="deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"
Jul 22 12:31:02.401431 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.401392    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb"} err="failed to get container status \"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb\": rpc error: code = NotFound desc = could not find container \"deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb\": container with ID starting with deb0ce45e3e5aa9ad09b393432969c154f051f437975a6aafb2a65c95d70a2cb not found: ID does not exist"
Jul 22 12:31:02.406164 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.406128    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:31:02.408395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:02.408373    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cf37a858-openshift-default-599d8b699c-bj5sz"]
Jul 22 12:31:04.133854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:04.133819    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" path="/var/lib/kubelet/pods/f6bcbe46-2453-4e6b-8444-70f9e686e971/volumes"
Jul 22 12:31:04.897986 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:04.897943    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:31:13.441471 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.441438    2565 generic.go:356] "Generic (PLEG): container finished" podID="ad56dccc-dd34-4508-b32f-456033007eca" containerID="f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08" exitCode=0
Jul 22 12:31:13.441893 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.441510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerDied","Data":"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"}
Jul 22 12:31:13.441951 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.441935    2565 scope.go:117] "RemoveContainer" containerID="f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"
Jul 22 12:31:13.442918 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.442890    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:31:13.442986 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.442959    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:31:13.443022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:13.442988    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:31:14.447833 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:14.447775    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerStarted","Data":"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"}
Jul 22 12:31:14.840229 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:14.840180    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:14.841667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:14.841622    2565 patch_prober.go:28] interesting pod/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 22 12:31:14.841847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:14.841691    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 22 12:31:15.372984 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:15.372941    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:15.841432 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:15.841382    2565 patch_prober.go:28] interesting pod/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 22 12:31:15.841974 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:15.841456    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 22 12:31:16.841360 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:16.841314    2565 patch_prober.go:28] interesting pod/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 22 12:31:16.841583 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:16.841388    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 22 12:31:17.841408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:17.841367    2565 patch_prober.go:28] interesting pod/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 22 12:31:17.841721 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:17.841681    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 22 12:31:18.840367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:18.840329    2565 patch_prober.go:28] interesting pod/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 22 12:31:18.840556 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:18.840388    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 22 12:31:19.844460 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:19.844430    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:19.845394 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:19.845377    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:24.652250 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652215    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"]
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652507    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652518    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652534    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652541    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652590    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.652623 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.652597    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bcbe46-2453-4e6b-8444-70f9e686e971" containerName="istio-proxy"
Jul 22 12:31:24.654178 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.654161    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:24.656293 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.656270    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:31:24.656509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.656493    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:31:24.663050 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.663023    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"]
Jul 22 12:31:24.742619 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.742576    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7\" (UID: \"072fa183-d61e-4c2f-9ae8-628a8885aa77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:24.843272 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.843231    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7\" (UID: \"072fa183-d61e-4c2f-9ae8-628a8885aa77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:24.843896 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.843877    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7\" (UID: \"072fa183-d61e-4c2f-9ae8-628a8885aa77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:24.964671 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:24.964583    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:25.092195 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:25.092114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"]
Jul 22 12:31:25.094507 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:31:25.094480    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072fa183_d61e_4c2f_9ae8_628a8885aa77.slice/crio-dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30 WatchSource:0}: Error finding container dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30: Status 404 returned error can't find the container with id dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30
Jul 22 12:31:25.501097 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:25.501005    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7" event={"ID":"072fa183-d61e-4c2f-9ae8-628a8885aa77","Type":"ContainerStarted","Data":"e0e4ac6374019098fe6c6aa3f50ec65b230088bdd37336bbe46f50d934a3cf9f"}
Jul 22 12:31:25.501097 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:25.501059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7" event={"ID":"072fa183-d61e-4c2f-9ae8-628a8885aa77","Type":"ContainerStarted","Data":"dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30"}
Jul 22 12:31:25.519343 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:25.519296    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7" podStartSLOduration=1.5192814110000001 podStartE2EDuration="1.519281411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:31:25.517919835 +0000 UTC m=+1805.941350890" watchObservedRunningTime="2026-07-22 12:31:25.519281411 +0000 UTC m=+1805.942712467"
Jul 22 12:31:26.507351 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:26.507312    2565 generic.go:356] "Generic (PLEG): container finished" podID="072fa183-d61e-4c2f-9ae8-628a8885aa77" containerID="e0e4ac6374019098fe6c6aa3f50ec65b230088bdd37336bbe46f50d934a3cf9f" exitCode=0
Jul 22 12:31:26.507734 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:26.507401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7" event={"ID":"072fa183-d61e-4c2f-9ae8-628a8885aa77","Type":"ContainerDied","Data":"e0e4ac6374019098fe6c6aa3f50ec65b230088bdd37336bbe46f50d934a3cf9f"}
Jul 22 12:31:27.640461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:27.640440    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:27.769482 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:27.769457    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca\") pod \"072fa183-d61e-4c2f-9ae8-628a8885aa77\" (UID: \"072fa183-d61e-4c2f-9ae8-628a8885aa77\") "
Jul 22 12:31:27.769820 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:27.769770    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "072fa183-d61e-4c2f-9ae8-628a8885aa77" (UID: "072fa183-d61e-4c2f-9ae8-628a8885aa77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:31:27.870064 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:27.870034    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072fa183-d61e-4c2f-9ae8-628a8885aa77-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:28.518501 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:28.518471    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"
Jul 22 12:31:28.518707 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:28.518473    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7" event={"ID":"072fa183-d61e-4c2f-9ae8-628a8885aa77","Type":"ContainerDied","Data":"dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30"}
Jul 22 12:31:28.518707 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:28.518595    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dda272ab1bcde41463f07e9b559c64c3ed51f8ec940a0b421d57ed8d4eec6c30"
Jul 22 12:31:30.212132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:30.212103    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:30.313927 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:30.313887    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:35.255289 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:35.255249    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:35.353909 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:35.353872    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:35.541469 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:35.541433    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:38.904699 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:38.904659    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:31:39.751081 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.751044    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"]
Jul 22 12:31:39.756086 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.756054    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:39.758211 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.758182    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a7ea4ba9-6lqt7"]
Jul 22 12:31:39.896182 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.896149    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:31:39.896385 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.896364    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy" containerID="cri-o://fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46" gracePeriod=30
Jul 22 12:31:39.918938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:39.918905    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:40.134739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:40.134700    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="072fa183-d61e-4c2f-9ae8-628a8885aa77" path="/var/lib/kubelet/pods/072fa183-d61e-4c2f-9ae8-628a8885aa77/volumes"
Jul 22 12:31:40.168486 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:40.168454    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:40.684823 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:40.684774    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:40.802182 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:40.802149    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:41.327639 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:41.327603    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:42.383348 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:42.383309    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:45.148930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.148899    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:45.217032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.216946    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.216979    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.216997    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217057    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217074    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2rnf\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217122    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217139    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217154    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket\") pod \"ad56dccc-dd34-4508-b32f-456033007eca\" (UID: \"ad56dccc-dd34-4508-b32f-456033007eca\") "
Jul 22 12:31:45.217312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217286    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:45.217715 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217512    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:45.217852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.217830    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:31:45.218109 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.218086    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:45.219095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.219070    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:45.229297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.229267    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:31:45.229297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.229287    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:31:45.229297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.229286    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf" (OuterVolumeSpecName: "kube-api-access-w2rnf") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "kube-api-access-w2rnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:31:45.229297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.229306    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad56dccc-dd34-4508-b32f-456033007eca" (UID: "ad56dccc-dd34-4508-b32f-456033007eca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:31:45.318371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318334    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad56dccc-dd34-4508-b32f-456033007eca-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318367    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318379    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318390    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318401    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318411    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2rnf\" (UniqueName: \"kubernetes.io/projected/ad56dccc-dd34-4508-b32f-456033007eca-kube-api-access-w2rnf\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318422    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318435    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad56dccc-dd34-4508-b32f-456033007eca-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.318611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.318449    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad56dccc-dd34-4508-b32f-456033007eca-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:31:45.330253 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.330223    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:45.595284 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.595254    2565 generic.go:356] "Generic (PLEG): container finished" podID="ad56dccc-dd34-4508-b32f-456033007eca" containerID="fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46" exitCode=0
Jul 22 12:31:45.595465 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.595291    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerDied","Data":"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"}
Jul 22 12:31:45.595465 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.595305    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"
Jul 22 12:31:45.595465 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.595332    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks" event={"ID":"ad56dccc-dd34-4508-b32f-456033007eca","Type":"ContainerDied","Data":"f910b75b8f0fa58e1952f333f6390b57aa4bb7ba43bea9b9bf5f70ae374cab6e"}
Jul 22 12:31:45.595465 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.595360    2565 scope.go:117] "RemoveContainer" containerID="fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"
Jul 22 12:31:45.604624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.604608    2565 scope.go:117] "RemoveContainer" containerID="f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"
Jul 22 12:31:45.612142 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.612123    2565 scope.go:117] "RemoveContainer" containerID="fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"
Jul 22 12:31:45.612384 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:31:45.612365    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46\": container with ID starting with fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46 not found: ID does not exist" containerID="fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"
Jul 22 12:31:45.612451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.612396    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46"} err="failed to get container status \"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46\": rpc error: code = NotFound desc = could not find container \"fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46\": container with ID starting with fd8bf456528a387251da5b6c273ca4d3f0780691f2298801cbcc376bb2144e46 not found: ID does not exist"
Jul 22 12:31:45.612451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.612419    2565 scope.go:117] "RemoveContainer" containerID="f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"
Jul 22 12:31:45.612701 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:31:45.612684    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08\": container with ID starting with f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08 not found: ID does not exist" containerID="f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"
Jul 22 12:31:45.612783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.612707    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08"} err="failed to get container status \"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08\": rpc error: code = NotFound desc = could not find container \"f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08\": container with ID starting with f9cb1cde7f33bf857fb004be275a2baa33693bd76906e7d526fe4b52eb66ea08 not found: ID does not exist"
Jul 22 12:31:45.622921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.622901    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:31:45.627086 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:45.627061    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a7ea4ba9-openshift-default-6496f8c-4dxks"]
Jul 22 12:31:46.133721 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:46.133684    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad56dccc-dd34-4508-b32f-456033007eca" path="/var/lib/kubelet/pods/ad56dccc-dd34-4508-b32f-456033007eca/volumes"
Jul 22 12:31:50.099839 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:50.099776    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:50.799553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:50.799492    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:50.899874 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:50.899839    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:50.995012 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:50.994973    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:51.092767 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:51.092679    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:31:51.570161 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:31:51.570114    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:17.572542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:17.572500    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:24.832218 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:24.832165    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:26.406489 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:26.406432    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:41.873060 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:41.873021    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:48.618171 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:48.618089    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:48.718784 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:48.718750    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:48.819493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:48.819453    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:48.923521 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:48.923420    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:32:59.398072 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:32:59.398038    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:00.263033 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:00.262988    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:08.034195 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:08.034133    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:08.098173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:08.098142    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:08.387331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:08.387244    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:08.484930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:08.484893    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:09.153132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:09.153100    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:09.271700 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:09.271667    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:09.348840 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:09.348806    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:10.824975 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:10.824936    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:11.018935 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:11.018896    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:13.496328 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:13.496284    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:33:19.229010 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:19.228969    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:19.330689 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:19.330653    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:19.943713 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:19.943676    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:22.901484 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:22.901443    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:33:26.364724 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.364693    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:26.469747 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.469719    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:26.629150 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.629066    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:26.939306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939228    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:33:26.939553 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939539    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939557    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939565    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939572    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939580    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="072fa183-d61e-4c2f-9ae8-628a8885aa77" containerName="revoke-keys"
Jul 22 12:33:26.939596 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939585    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="072fa183-d61e-4c2f-9ae8-628a8885aa77" containerName="revoke-keys"
Jul 22 12:33:26.939750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939633    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939641    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad56dccc-dd34-4508-b32f-456033007eca" containerName="istio-proxy"
Jul 22 12:33:26.939750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.939648    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="072fa183-d61e-4c2f-9ae8-628a8885aa77" containerName="revoke-keys"
Jul 22 12:33:26.942843 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.942822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:26.944982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.944966    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9ece34e1-openshift-default-dockercfg-bzlfd\""
Jul 22 12:33:26.957546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.957521    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:26.962420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:26.962399    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:33:27.049049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049011    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049049 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049202    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049289    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.049660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.049404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4v8d\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150115 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150115 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150348 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150348 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150166    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150346    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4v8d\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150566 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150566 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150566 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150516    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150566 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150769 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150598    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150769 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150699    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.150900 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150783    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.151003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.150981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.152823 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.152783    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.152969 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.152951    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.158542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.158520    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.158850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.158822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4v8d\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d\") pod \"e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") " pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.172311 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.172285    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:27.253115 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.253036    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:27.390720 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.390674    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:33:27.393475 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:33:27.393446    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930402e6_6fd6_4b5b_b69b_2f913402dac0.slice/crio-12224b776ac1c65d087805182dcfe86b4486045149f4fd064407a8f4705b6cbe WatchSource:0}: Error finding container 12224b776ac1c65d087805182dcfe86b4486045149f4fd064407a8f4705b6cbe: Status 404 returned error can't find the container with id 12224b776ac1c65d087805182dcfe86b4486045149f4fd064407a8f4705b6cbe
Jul 22 12:33:27.395593 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.395557    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:27.395685 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.395641    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:27.395685 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:27.395670    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:28.062291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:28.062254    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerStarted","Data":"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"}
Jul 22 12:33:28.062484 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:28.062336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerStarted","Data":"12224b776ac1c65d087805182dcfe86b4486045149f4fd064407a8f4705b6cbe"}
Jul 22 12:33:28.098665 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:28.098608    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podStartSLOduration=2.098590752 podStartE2EDuration="2.098590752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:33:28.097146909 +0000 UTC m=+1928.520577968" watchObservedRunningTime="2026-07-22 12:33:28.098590752 +0000 UTC m=+1928.522021807"
Jul 22 12:33:28.253626 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:28.253585    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:28.258073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:28.258048    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:29.067173 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:29.067137    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:29.067966 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:29.067947    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:30.261776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:30.261706    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:30.360446 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:30.360412    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:30.431413 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:30.431376    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:30.532887 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:30.532848    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:31.069978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:31.069941    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:31.171346 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:31.171308    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:34.702255 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:34.702223    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:35.347955 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:35.347908    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:35.449167 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:35.449125    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:35.830822 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:35.830767    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:36.796772 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:36.796734    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:33:42.159405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.159351    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:42.218993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.218949    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:42.579639 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.579591    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"]
Jul 22 12:33:42.583125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.583104    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:42.585342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.585319    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:33:42.585475 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.585345    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:33:42.591765 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.591191    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"]
Jul 22 12:33:42.691149 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.691103    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx\" (UID: \"46940f18-4924-477b-8082-022806cfccba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:42.791767 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.791722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx\" (UID: \"46940f18-4924-477b-8082-022806cfccba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:42.792382 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.792362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx\" (UID: \"46940f18-4924-477b-8082-022806cfccba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:42.896112 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:42.896039    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:43.030450 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:43.030332    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"]
Jul 22 12:33:43.033145 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:33:43.033109    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46940f18_4924_477b_8082_022806cfccba.slice/crio-7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197 WatchSource:0}: Error finding container 7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197: Status 404 returned error can't find the container with id 7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197
Jul 22 12:33:43.130755 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:43.130727    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx" event={"ID":"46940f18-4924-477b-8082-022806cfccba","Type":"ContainerStarted","Data":"7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197"}
Jul 22 12:33:44.136461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:44.136427    2565 generic.go:356] "Generic (PLEG): container finished" podID="46940f18-4924-477b-8082-022806cfccba" containerID="916787a2beea18ee2a5e166c037412f6c0a6a3edc8b6b3d9c75e1777627f5f3a" exitCode=0
Jul 22 12:33:44.136896 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:44.136485    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx" event={"ID":"46940f18-4924-477b-8082-022806cfccba","Type":"ContainerDied","Data":"916787a2beea18ee2a5e166c037412f6c0a6a3edc8b6b3d9c75e1777627f5f3a"}
Jul 22 12:33:44.251972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:44.251403    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:35244->10.132.0.35:15021: read: connection reset by peer" start-of-body=
Jul 22 12:33:44.251972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:44.251469    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:35244->10.132.0.35:15021: read: connection reset by peer"
Jul 22 12:33:45.142341 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.142306    2565 generic.go:356] "Generic (PLEG): container finished" podID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerID="7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1" exitCode=0
Jul 22 12:33:45.142737 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.142379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerDied","Data":"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"}
Jul 22 12:33:45.143095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.142849    2565 scope.go:117] "RemoveContainer" containerID="7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"
Jul 22 12:33:45.143845 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.143810    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:45.143965 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.143897    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:45.143965 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.143948    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:33:45.294143 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.294120    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:45.417933 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.417847    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca\") pod \"46940f18-4924-477b-8082-022806cfccba\" (UID: \"46940f18-4924-477b-8082-022806cfccba\") "
Jul 22 12:33:45.418189 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.418166    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46940f18-4924-477b-8082-022806cfccba" (UID: "46940f18-4924-477b-8082-022806cfccba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:33:45.518647 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:45.518614    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46940f18-4924-477b-8082-022806cfccba-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:33:46.148121 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.148090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerStarted","Data":"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"}
Jul 22 12:33:46.149318 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.149295    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx" event={"ID":"46940f18-4924-477b-8082-022806cfccba","Type":"ContainerDied","Data":"7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197"}
Jul 22 12:33:46.149413 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.149321    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"
Jul 22 12:33:46.149413 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.149332    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df01ca2844c262f5dc896f5c441764180c2e82a4dd8d4812c0349084602f197"
Jul 22 12:33:46.253711 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.253676    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:46.253928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.253857    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:46.253928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:46.253908    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:47.253849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:47.253782    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:47.254228 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:47.253870    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:48.254022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:48.253982    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:48.254606 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:48.254571    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:49.255226 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:49.255190    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:49.255665 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:49.255245    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:50.254384 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:50.254338    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:50.254568 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:50.254405    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:51.253595 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:51.253557    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 22 12:33:51.254216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:51.254185    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 22 12:33:52.257386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:52.257355    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:52.257782 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:52.257723    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:52.258370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:52.258351    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:33:53.118739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:53.118705    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:55.999398 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:55.999365    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:33:57.663128 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:57.663092    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:57.668506 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:57.668480    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"]
Jul 22 12:33:57.672282 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:57.672261    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9ece34e1-cqckx"]
Jul 22 12:33:58.081695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:58.081661    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:33:58.133537 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:33:58.133508    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46940f18-4924-477b-8082-022806cfccba" path="/var/lib/kubelet/pods/46940f18-4924-477b-8082-022806cfccba/volumes"
Jul 22 12:34:04.717734 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:04.717697    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:04.817734 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:04.817704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:04.997464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:04.997381    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:34:04.997616 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:04.997583    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" containerID="cri-o://a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26" gracePeriod=30
Jul 22 12:34:05.027993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:05.027964    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:05.253854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:05.253763    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:05.381184 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:05.381152    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:05.479834 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:05.479776    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:07.101433 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:07.101400    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:07.259863 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:07.259830    2565 patch_prober.go:28] interesting pod/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:34:07.260045 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:07.259890    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:34:10.245038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.245015    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:34:10.258710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.258646    2565 generic.go:356] "Generic (PLEG): container finished" podID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerID="a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26" exitCode=0
Jul 22 12:34:10.258710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.258695    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"
Jul 22 12:34:10.258710 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.258706    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerDied","Data":"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"}
Jul 22 12:34:10.258914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.258748    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj" event={"ID":"930402e6-6fd6-4b5b-b69b-2f913402dac0","Type":"ContainerDied","Data":"12224b776ac1c65d087805182dcfe86b4486045149f4fd064407a8f4705b6cbe"}
Jul 22 12:34:10.258914 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.258768    2565 scope.go:117] "RemoveContainer" containerID="a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"
Jul 22 12:34:10.267471 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.267395    2565 scope.go:117] "RemoveContainer" containerID="7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"
Jul 22 12:34:10.275703 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.275687    2565 scope.go:117] "RemoveContainer" containerID="a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"
Jul 22 12:34:10.275979 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:34:10.275957    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26\": container with ID starting with a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26 not found: ID does not exist" containerID="a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"
Jul 22 12:34:10.276077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.275984    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26"} err="failed to get container status \"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26\": rpc error: code = NotFound desc = could not find container \"a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26\": container with ID starting with a55a388a361f526d71d3c017094a1e43e3d103b640816396131a345b4023dd26 not found: ID does not exist"
Jul 22 12:34:10.276077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.276000    2565 scope.go:117] "RemoveContainer" containerID="7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"
Jul 22 12:34:10.276247 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:34:10.276231    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1\": container with ID starting with 7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1 not found: ID does not exist" containerID="7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"
Jul 22 12:34:10.276290 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.276251    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1"} err="failed to get container status \"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1\": rpc error: code = NotFound desc = could not find container \"7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1\": container with ID starting with 7c987e9cabf6580dac6bd65399dc6a301529cb2be981908556845d60586bb7f1 not found: ID does not exist"
Jul 22 12:34:10.319664 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319628    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319664 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319668    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319692    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4v8d\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319727    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319776    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319815    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.319852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319843    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.320105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319889    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.320105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.319920    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs\") pod \"930402e6-6fd6-4b5b-b69b-2f913402dac0\" (UID: \"930402e6-6fd6-4b5b-b69b-2f913402dac0\") "
Jul 22 12:34:10.320216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.320123    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:34:10.320532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.320483    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:34:10.320532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.320506    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:34:10.320698 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.320594    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:34:10.320758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.320697    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data" (OuterVolumeSpecName: "istio-data") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:34:10.321997 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.321953    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:34:10.322075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.321997    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d" (OuterVolumeSpecName: "kube-api-access-m4v8d") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "kube-api-access-m4v8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:34:10.322075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.322017    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:34:10.322075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.322056    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token" (OuterVolumeSpecName: "istio-token") pod "930402e6-6fd6-4b5b-b69b-2f913402dac0" (UID: "930402e6-6fd6-4b5b-b69b-2f913402dac0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:34:10.420712 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420677    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420712 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420706    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420712 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420716    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/930402e6-6fd6-4b5b-b69b-2f913402dac0-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420725    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420735    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420744    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420751    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420759    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4v8d\" (UniqueName: \"kubernetes.io/projected/930402e6-6fd6-4b5b-b69b-2f913402dac0-kube-api-access-m4v8d\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.420963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.420766    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/930402e6-6fd6-4b5b-b69b-2f913402dac0-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:34:10.583236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.583206    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:34:10.588373 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:10.588348    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9ece34e1-openshift-default-6b89bf58f9-2bszj"]
Jul 22 12:34:12.121671 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.121636    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:12.138464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.138304    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" path="/var/lib/kubelet/pods/930402e6-6fd6-4b5b-b69b-2f913402dac0/volumes"
Jul 22 12:34:12.197075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.197043    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:12.269740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.269698    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:12.370989 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.370951    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:12.489653 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:12.489552    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:19.019133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:19.019095    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:19.119842 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:19.119806    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:26.001176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.001134    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:26.244443 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244408    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:34:26.244722 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244708    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46940f18-4924-477b-8082-022806cfccba" containerName="revoke-keys"
Jul 22 12:34:26.244722 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244723    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="46940f18-4924-477b-8082-022806cfccba" containerName="revoke-keys"
Jul 22 12:34:26.244826 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244742    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.244826 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244748    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.244826 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244753    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.244826 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244760    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.244970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244831    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.244970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244838    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="46940f18-4924-477b-8082-022806cfccba" containerName="revoke-keys"
Jul 22 12:34:26.244970 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.244844    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="930402e6-6fd6-4b5b-b69b-2f913402dac0" containerName="istio-proxy"
Jul 22 12:34:26.247682 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.247657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.249722 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.249702    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a70c8883-openshift-default-dockercfg-sckg8\""
Jul 22 12:34:26.259419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.259394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:34:26.281182 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.281151    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:26.342229 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342195    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-692xz\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342388    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342466    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342652 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.342652 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.342596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443622 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443585    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443622 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443757    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692xz\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.443928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.443957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444010    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444078    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444379 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444219    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444379 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444379 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444341    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.444592 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.444571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.446114 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.446095    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.446381 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.446364    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.452592 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.452566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-692xz\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.453137 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.453115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token\") pod \"e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") " pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.557338 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.557303    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:26.630931 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.630898    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:26.693821 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.693774    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:34:26.696861 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:34:26.696833    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad09c5f_bc5a_4832_b591_9c29ec6c6af3.slice/crio-a72b856f69dd512a5f53f389e4b2fa69c5c3d1fb875b01caa8b67b0ed0d75d4d WatchSource:0}: Error finding container a72b856f69dd512a5f53f389e4b2fa69c5c3d1fb875b01caa8b67b0ed0d75d4d: Status 404 returned error can't find the container with id a72b856f69dd512a5f53f389e4b2fa69c5c3d1fb875b01caa8b67b0ed0d75d4d
Jul 22 12:34:26.698807 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.698764    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:26.698868 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.698840    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:26.698916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:26.698893    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:27.345137 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:27.345099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerStarted","Data":"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"}
Jul 22 12:34:27.345525 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:27.345155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerStarted","Data":"a72b856f69dd512a5f53f389e4b2fa69c5c3d1fb875b01caa8b67b0ed0d75d4d"}
Jul 22 12:34:27.367291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:27.367236    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podStartSLOduration=1.367219765 podStartE2EDuration="1.367219765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:34:27.365985352 +0000 UTC m=+1987.789416404" watchObservedRunningTime="2026-07-22 12:34:27.367219765 +0000 UTC m=+1987.790650887"
Jul 22 12:34:27.558405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:27.558367    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:27.563618 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:27.563594    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:28.349998 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:28.349966    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:28.350869 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:28.350850    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:29.324673 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:29.324626    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:29.424735 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:29.424689    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:29.566859 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:29.566822    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:29.804637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:29.804601    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:30.176855 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:30.176738    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:30.272501 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:30.272464    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:33.844225 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:33.844186    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:34.002929 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:34.002890    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:34:42.419435 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.419402    2565 generic.go:356] "Generic (PLEG): container finished" podID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerID="e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464" exitCode=0
Jul 22 12:34:42.419882 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.419452    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerDied","Data":"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"}
Jul 22 12:34:42.419882 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.419783    2565 scope.go:117] "RemoveContainer" containerID="e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"
Jul 22 12:34:42.420728 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.420699    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:42.420805 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.420764    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:42.420855 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:42.420814    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:34:43.350881 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:43.350847    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:43.426206 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:43.426173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerStarted","Data":"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"}
Jul 22 12:34:43.557730 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:43.557685    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:43.559032 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:43.559003    2565 patch_prober.go:28] interesting pod/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 22 12:34:43.559191 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:43.559051    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 22 12:34:44.558106 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:44.558059    2565 patch_prober.go:28] interesting pod/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 22 12:34:44.558610 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:44.558120    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 22 12:34:45.558450 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:45.558407    2565 patch_prober.go:28] interesting pod/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 22 12:34:45.558881 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:45.558477    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 22 12:34:46.558347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:46.558306    2565 patch_prober.go:28] interesting pod/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 22 12:34:46.558536 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:46.558365    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 22 12:34:47.561514 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:47.561476    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:47.562003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:47.561817    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:47.562532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:47.562511    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:34:56.797945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.797905    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:34:56.801219 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.801199    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.807849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.804250    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a70c8883\"/\"default-dockercfg-rhmx5\""
Jul 22 12:34:56.807849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.804504    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a70c8883\"/\"e2e-lifecycle-model-a70c8883-kserve-self-signed-certs\""
Jul 22 12:34:56.807849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.804819    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a70c8883\"/\"openshift-service-ca.crt\""
Jul 22 12:34:56.807849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.805033    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a70c8883\"/\"kube-root-ca.crt\""
Jul 22 12:34:56.814600 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.814576    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:34:56.862321 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.862280    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:34:56.896937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.896910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.897084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.896960    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.897084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.897010    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.897084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.897042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.897084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.897066    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.897238 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.897083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wqg\" (UniqueName: \"kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.997783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.997783 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.998054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.998054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.998054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997916    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.998054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.997943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wqg\" (UniqueName: \"kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.998427 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.998385    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.999109 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.998753    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:56.999109 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:56.999058    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:57.001703 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.001678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:57.002446 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.002376    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:57.006881 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.006855    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wqg\" (UniqueName: \"kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg\") pod \"e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") " pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:57.115844 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.115732    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:34:57.263571 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.263543    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:34:57.265596 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:34:57.265568    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc03fd370_c6eb_4ef9_bda0_7d6bf12e7677.slice/crio-cd2c9329c1677437104edce7c963242e179187c254a0fbff65db1779fe0877ff WatchSource:0}: Error finding container cd2c9329c1677437104edce7c963242e179187c254a0fbff65db1779fe0877ff: Status 404 returned error can't find the container with id cd2c9329c1677437104edce7c963242e179187c254a0fbff65db1779fe0877ff
Jul 22 12:34:57.491847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.491737    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" event={"ID":"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677","Type":"ContainerStarted","Data":"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"}
Jul 22 12:34:57.491847 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.491778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" event={"ID":"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677","Type":"ContainerStarted","Data":"cd2c9329c1677437104edce7c963242e179187c254a0fbff65db1779fe0877ff"}
Jul 22 12:34:57.510531 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:57.510483    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" podStartSLOduration=1.510467969 podStartE2EDuration="1.510467969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:34:57.509057314 +0000 UTC m=+2017.932488369" watchObservedRunningTime="2026-07-22 12:34:57.510467969 +0000 UTC m=+2017.933899064"
Jul 22 12:34:59.207125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:34:59.207092    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:35:00.311239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:00.311203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:35:07.116321 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:07.116276    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:35:07.116822 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:07.116366    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:35:07.128862 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:07.128838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:35:07.555353 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:07.555325    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:35:19.407103 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:19.407061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:35:20.248849 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:20.248819    2565 scope.go:117] "RemoveContainer" containerID="726046d5ab4a8b5bd8a1ce92da6c88b30a7599c55ff5e49d6360d39091eceb8e"
Jul 22 12:35:22.797359 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.797317    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"]
Jul 22 12:35:22.801972 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.801949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:22.804291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.804268    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:35:22.804388 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.804320    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:35:22.809008 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.808984    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"]
Jul 22 12:35:22.911052 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:22.911016    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p\" (UID: \"314b6482-d038-4826-9013-a0544296f3ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:23.012077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.012041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p\" (UID: \"314b6482-d038-4826-9013-a0544296f3ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:23.012681 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.012652    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p\" (UID: \"314b6482-d038-4826-9013-a0544296f3ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:23.022764 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.022732    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:35:23.113312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.113230    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:23.246318 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.246289    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"]
Jul 22 12:35:23.250372 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:35:23.250345    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314b6482_d038_4826_9013_a0544296f3ab.slice/crio-13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7 WatchSource:0}: Error finding container 13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7: Status 404 returned error can't find the container with id 13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7
Jul 22 12:35:23.624708 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.624672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p" event={"ID":"314b6482-d038-4826-9013-a0544296f3ab","Type":"ContainerStarted","Data":"d6c243921ac0e2c413a97b9f217c1e14fee1d408bd5b3fb76dd53246d2022414"}
Jul 22 12:35:23.624889 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.624733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p" event={"ID":"314b6482-d038-4826-9013-a0544296f3ab","Type":"ContainerStarted","Data":"13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7"}
Jul 22 12:35:23.642087 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:23.642038    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p" podStartSLOduration=1.642022136 podStartE2EDuration="1.642022136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:35:23.640864661 +0000 UTC m=+2044.064295718" watchObservedRunningTime="2026-07-22 12:35:23.642022136 +0000 UTC m=+2044.065453191"
Jul 22 12:35:24.630930 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:24.630893    2565 generic.go:356] "Generic (PLEG): container finished" podID="314b6482-d038-4826-9013-a0544296f3ab" containerID="d6c243921ac0e2c413a97b9f217c1e14fee1d408bd5b3fb76dd53246d2022414" exitCode=0
Jul 22 12:35:24.631320 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:24.630975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p" event={"ID":"314b6482-d038-4826-9013-a0544296f3ab","Type":"ContainerDied","Data":"d6c243921ac0e2c413a97b9f217c1e14fee1d408bd5b3fb76dd53246d2022414"}
Jul 22 12:35:25.762979 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:25.762955    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:25.836071 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:25.836036    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca\") pod \"314b6482-d038-4826-9013-a0544296f3ab\" (UID: \"314b6482-d038-4826-9013-a0544296f3ab\") "
Jul 22 12:35:25.836425 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:25.836403    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "314b6482-d038-4826-9013-a0544296f3ab" (UID: "314b6482-d038-4826-9013-a0544296f3ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:35:25.937529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:25.937446    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314b6482-d038-4826-9013-a0544296f3ab-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:35:26.641399 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:26.641361    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p" event={"ID":"314b6482-d038-4826-9013-a0544296f3ab","Type":"ContainerDied","Data":"13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7"}
Jul 22 12:35:26.641399 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:26.641387    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"
Jul 22 12:35:26.641662 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:26.641438    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13bdd42faae4b5ad23e96063cbabb07f2621ce70515f00833d1e011cc360c7d7"
Jul 22 12:35:36.904140 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:36.904106    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:35:38.420458 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:38.420420    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:35:42.906219 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:42.906137    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:35:42.914511 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:42.914484    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"]
Jul 22 12:35:42.922817 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:42.922776    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a70c8883-jlv4p"]
Jul 22 12:35:44.133632 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:44.133600    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314b6482-d038-4826-9013-a0544296f3ab" path="/var/lib/kubelet/pods/314b6482-d038-4826-9013-a0544296f3ab/volumes"
Jul 22 12:35:45.215081 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:45.215041    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:35:45.710828 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:45.710758    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:35:50.198551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:50.198515    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:35:50.294210 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:35:50.294174    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:05.218692 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.218620    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" podUID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" containerName="main" containerID="cri-o://b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6" gracePeriod=45
Jul 22 12:36:05.478127 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.478067    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:36:05.482690 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482670    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.482770 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482702    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.482770 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482728    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.482770 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482749    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.482931 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482777    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5wqg\" (UniqueName: \"kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.482931 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482822    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache\") pod \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\" (UID: \"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677\") "
Jul 22 12:36:05.483040 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.482970    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home" (OuterVolumeSpecName: "home") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:05.483115 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.483088    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:05.483215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.483187    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache" (OuterVolumeSpecName: "model-cache") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:05.484993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.484970    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm" (OuterVolumeSpecName: "dshm") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:05.484993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.484986    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:36:05.485138 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.485042    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg" (OuterVolumeSpecName: "kube-api-access-b5wqg") pod "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" (UID: "c03fd370-c6eb-4ef9-bda0-7d6bf12e7677"). InnerVolumeSpecName "kube-api-access-b5wqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:36:05.584280 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584243    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5wqg\" (UniqueName: \"kubernetes.io/projected/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-kube-api-access-b5wqg\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.584280 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584275    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.584280 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584285    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.584509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584292    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.584509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584303    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.584509 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.584314    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:05.826766 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.826734    2565 generic.go:356] "Generic (PLEG): container finished" podID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" containerID="b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6" exitCode=0
Jul 22 12:36:05.826947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.826780    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"
Jul 22 12:36:05.826947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.826813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" event={"ID":"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677","Type":"ContainerDied","Data":"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"}
Jul 22 12:36:05.826947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.826857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf" event={"ID":"c03fd370-c6eb-4ef9-bda0-7d6bf12e7677","Type":"ContainerDied","Data":"cd2c9329c1677437104edce7c963242e179187c254a0fbff65db1779fe0877ff"}
Jul 22 12:36:05.826947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.826876    2565 scope.go:117] "RemoveContainer" containerID="b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"
Jul 22 12:36:05.836035 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.836018    2565 scope.go:117] "RemoveContainer" containerID="b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"
Jul 22 12:36:05.836302 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:36:05.836283    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6\": container with ID starting with b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6 not found: ID does not exist" containerID="b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"
Jul 22 12:36:05.836351 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.836312    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6"} err="failed to get container status \"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6\": rpc error: code = NotFound desc = could not find container \"b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6\": container with ID starting with b930f2083c8072cf50e405f044bc9368ae3aa09ffcbccb79ac01468eb5f09ef6 not found: ID does not exist"
Jul 22 12:36:05.849214 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.849184    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:36:05.853848 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:05.853826    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a70c8883/e2e-lifecycle-model-a70c8883-kserve-6b7c995899-5szqf"]
Jul 22 12:36:06.134101 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:06.134022    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" path="/var/lib/kubelet/pods/c03fd370-c6eb-4ef9-bda0-7d6bf12e7677/volumes"
Jul 22 12:36:20.271367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:20.267720    2565 scope.go:117] "RemoveContainer" containerID="8b625c647f90215ed41dd921a3b55be4814ef52d068c09fde2f06f722bbe69e5"
Jul 22 12:36:20.280542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:20.280519    2565 scope.go:117] "RemoveContainer" containerID="2af1069450b83fcc0ded78f20d336527478869476559a81ce9c3309179be8634"
Jul 22 12:36:26.352016 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:26.351984    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:28.053709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:28.053650    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:31.032408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:31.032369    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:31.133126 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:31.133091    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:34.762825 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:34.762770    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:34.858076 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:34.858036    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:35.078545 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:35.078509    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:36:35.078824 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:35.078771    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy" containerID="cri-o://1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8" gracePeriod=30
Jul 22 12:36:35.111441 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:35.111408    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:35.339917 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:35.339819    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:36.360127 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:36.360097    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:36.455621 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:36.455588    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:36.559885 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:36.559843    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:37.304674 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:37.304632    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:37.533910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:37.533874    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:38.834159 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:38.834125    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:39.102312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:39.102231    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:36:40.328851 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.328827    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:36:40.471611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471517    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471568    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-692xz\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471600    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471637    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471681    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471728    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471750    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471782    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.471928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.471852    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo\") pod \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\" (UID: \"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3\") "
Jul 22 12:36:40.472216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.472067    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:40.472216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.472077    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:40.472216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.472101    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:40.472414 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.472387    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:36:40.472703 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.472678    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:40.473779 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.473757    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:36:40.474069 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.474047    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:36:40.474325 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.474301    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz" (OuterVolumeSpecName: "kube-api-access-692xz") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "kube-api-access-692xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:36:40.474397 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.474303    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" (UID: "7ad09c5f-bc5a-4832-b591-9c29ec6c6af3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:36:40.572418 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572375    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572418 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572414    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572430    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572441    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572452    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572462    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572472    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-692xz\" (UniqueName: \"kubernetes.io/projected/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-kube-api-access-692xz\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572483    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.572649 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.572496    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:36:40.992570 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.992538    2565 generic.go:356] "Generic (PLEG): container finished" podID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerID="1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8" exitCode=0
Jul 22 12:36:40.992732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.992585    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"
Jul 22 12:36:40.992732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.992597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerDied","Data":"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"}
Jul 22 12:36:40.992732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.992641    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5" event={"ID":"7ad09c5f-bc5a-4832-b591-9c29ec6c6af3","Type":"ContainerDied","Data":"a72b856f69dd512a5f53f389e4b2fa69c5c3d1fb875b01caa8b67b0ed0d75d4d"}
Jul 22 12:36:40.992732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:40.992668    2565 scope.go:117] "RemoveContainer" containerID="1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"
Jul 22 12:36:41.006879 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.006858    2565 scope.go:117] "RemoveContainer" containerID="e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"
Jul 22 12:36:41.014511 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.014487    2565 scope.go:117] "RemoveContainer" containerID="1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"
Jul 22 12:36:41.014775 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:36:41.014755    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8\": container with ID starting with 1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8 not found: ID does not exist" containerID="1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"
Jul 22 12:36:41.014897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.014800    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8"} err="failed to get container status \"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8\": rpc error: code = NotFound desc = could not find container \"1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8\": container with ID starting with 1f4ae369219e54efa844053e50fec6d2d166ee932f7d3edbf9aa89a9792629e8 not found: ID does not exist"
Jul 22 12:36:41.014897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.014825    2565 scope.go:117] "RemoveContainer" containerID="e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"
Jul 22 12:36:41.015096 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:36:41.015078    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464\": container with ID starting with e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464 not found: ID does not exist" containerID="e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"
Jul 22 12:36:41.015136 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.015102    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464"} err="failed to get container status \"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464\": rpc error: code = NotFound desc = could not find container \"e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464\": container with ID starting with e75f584718ab1e5ff61b8c19e19289c2f83b59ae682e31a7918ecf807509e464 not found: ID does not exist"
Jul 22 12:36:41.035078 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.035050    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:36:41.043519 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:41.043498    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a70c8883-openshift-default-66cc54bd9f-s5lx5"]
Jul 22 12:36:42.133052 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:42.133021    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" path="/var/lib/kubelet/pods/7ad09c5f-bc5a-4832-b591-9c29ec6c6af3/volumes"
Jul 22 12:36:45.579540 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:45.579501    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:45.647361 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:45.647328    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:46.284260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:46.284220    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:49.207899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:49.207865    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:36:50.354692 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:50.354656    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:53.718978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:53.718943    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:53.828603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:53.828563    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:36:53.960023 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:36:53.959985    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:04.479118 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:04.479079    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:04.793439 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:04.793407    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:20.305214 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:20.305146    2565 scope.go:117] "RemoveContainer" containerID="ea7487c2e347f76010651edf7ec114732330b76660cb8e860e0e6859367ddecf"
Jul 22 12:37:25.824167 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:25.824131    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.144959 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.144918    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.763921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.763889    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:37:33.764195 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764183    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.764241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764197    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.764241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764208    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.764241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764213    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.764241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764221    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314b6482-d038-4826-9013-a0544296f3ab" containerName="revoke-keys"
Jul 22 12:37:33.764241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764227    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="314b6482-d038-4826-9013-a0544296f3ab" containerName="revoke-keys"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764253    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" containerName="main"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764259    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" containerName="main"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764306    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c03fd370-c6eb-4ef9-bda0-7d6bf12e7677" containerName="main"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764312    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="314b6482-d038-4826-9013-a0544296f3ab" containerName="revoke-keys"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764319    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.764416 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.764325    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad09c5f-bc5a-4832-b591-9c29ec6c6af3" containerName="istio-proxy"
Jul 22 12:37:33.766968 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.766951    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.769135 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.769113    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-5e8367-openshift-default-dockercfg-fcbzh\""
Jul 22 12:37:33.779327 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.779301    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:37:33.897141 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897108    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897173    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897302    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897364    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh72b\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.897555 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.897417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998181    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998243    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998268    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh72b\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998730 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998391    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998730 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998691    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998730 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998720    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.998897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998770    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.999015 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.998992    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:33.999274 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:33.999257    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.000611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.000589    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.000764 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.000748    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.007213 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.007186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.007401 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.007382    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh72b\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b\") pod \"e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") " pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.076290 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.076256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:34.213085 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.213057    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:37:34.215592 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:37:34.215565    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e88a9f_1fa4_4251_a1f1_da913e8533fb.slice/crio-00ca83d4f8675a3fba0ba7fce2a7e9599419f838ccd9f372d1dff37f3c9020c3 WatchSource:0}: Error finding container 00ca83d4f8675a3fba0ba7fce2a7e9599419f838ccd9f372d1dff37f3c9020c3: Status 404 returned error can't find the container with id 00ca83d4f8675a3fba0ba7fce2a7e9599419f838ccd9f372d1dff37f3c9020c3
Jul 22 12:37:34.218066 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.218032    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:34.218168 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.218112    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:34.218168 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.218159    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:34.238156 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.238129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerStarted","Data":"00ca83d4f8675a3fba0ba7fce2a7e9599419f838ccd9f372d1dff37f3c9020c3"}
Jul 22 12:37:34.946393 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:34.946354    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:35.244319 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:35.244238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerStarted","Data":"d103103ae11da16ef0d896e4f1fcdc3ed78a8a163e3dab64c6b89ee0b64be613"}
Jul 22 12:37:35.276278 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:35.276228    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podStartSLOduration=2.276212609 podStartE2EDuration="2.276212609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:35.275024328 +0000 UTC m=+2175.698455377" watchObservedRunningTime="2026-07-22 12:37:35.276212609 +0000 UTC m=+2175.699643738"
Jul 22 12:37:36.077051 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:36.077012    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:36.082163 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:36.082133    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:36.249163 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:36.249122    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:36.250023 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:36.249998    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:37.270761 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:37.270728    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:37.375483 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:37.375446    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:37.472624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:37.472583    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:37.750020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:37.749886    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:38.101535 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:38.101479    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:38.202481 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:38.202448    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:41.723011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:41.722975    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:42.061169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.061133    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:42.354643 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.354571    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:37:42.358003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.357982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.360332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.360310    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-07497a-openshift-default-dockercfg-p2wwd\""
Jul 22 12:37:42.372936 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.372915    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:37:42.385575 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.385549    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:42.469816 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469765    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgff\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469951    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.469998    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.470022    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.470074    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.470260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.470129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.517637 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.517601    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:42.571041 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571005    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgff\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571146    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571236 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571627 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571278    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571627 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571449    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571736 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571647    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571736 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571692    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.571867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571840    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.572001 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.571977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.574644 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.574618    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.574859 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.574834    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.581002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.580978    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.581162 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.581142    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgff\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff\") pod \"e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") " pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.668234 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.668141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:42.802729 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.802696    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:37:42.807420 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.807399    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:37:42.809353 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:37:42.809321    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f68167_db52_4110_9e81_a147cdc8082f.slice/crio-439866fe411a107bd7944597eda4abcd6d84ba2b36c035bcd8008f546262b8c1 WatchSource:0}: Error finding container 439866fe411a107bd7944597eda4abcd6d84ba2b36c035bcd8008f546262b8c1: Status 404 returned error can't find the container with id 439866fe411a107bd7944597eda4abcd6d84ba2b36c035bcd8008f546262b8c1
Jul 22 12:37:42.811449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.811418    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:42.811567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.811513    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:42.811634 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:42.811573    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:43.281076 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:43.281038    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerStarted","Data":"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"}
Jul 22 12:37:43.281274 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:43.281091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerStarted","Data":"439866fe411a107bd7944597eda4abcd6d84ba2b36c035bcd8008f546262b8c1"}
Jul 22 12:37:43.311105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:43.311054    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podStartSLOduration=1.311040003 podStartE2EDuration="1.311040003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:43.308414488 +0000 UTC m=+2183.731845543" watchObservedRunningTime="2026-07-22 12:37:43.311040003 +0000 UTC m=+2183.734471058"
Jul 22 12:37:43.668733 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:43.668643    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:43.673241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:43.673208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:44.286648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:44.286615    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:44.287528 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:44.287505    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:37:45.216912 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:45.216873    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:45.311297 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:45.311258    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:45.453409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:45.453351    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:45.661747 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:45.661705    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:46.046624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:46.046583    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:46.151524 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:46.151489    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:49.704911 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:49.704871    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:51.250185 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.250154    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 22 12:37:51.250578 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.250205    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 22 12:37:51.321880 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.321852    2565 generic.go:356] "Generic (PLEG): container finished" podID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerID="d103103ae11da16ef0d896e4f1fcdc3ed78a8a163e3dab64c6b89ee0b64be613" exitCode=0
Jul 22 12:37:51.322056 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.321929    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerDied","Data":"d103103ae11da16ef0d896e4f1fcdc3ed78a8a163e3dab64c6b89ee0b64be613"}
Jul 22 12:37:51.322318 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.322303    2565 scope.go:117] "RemoveContainer" containerID="d103103ae11da16ef0d896e4f1fcdc3ed78a8a163e3dab64c6b89ee0b64be613"
Jul 22 12:37:51.323302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.323270    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:51.323419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.323336    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:51.323419 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.323367    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:37:51.803189 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:51.803156    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:37:52.328628 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:52.328575    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerStarted","Data":"3fc368122383a490e3e3d514f2d9bc26b04b6cf83a96a9faa5172a17f31937d5"}
Jul 22 12:37:53.076811 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:53.076751    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:53.077252 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:53.077217    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 22 12:37:53.077347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:53.077280    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 22 12:37:54.077752 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:54.077711    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 22 12:37:54.078773 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:54.078348    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 22 12:37:55.077330 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:55.077287    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 22 12:37:55.077532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:55.077362    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 22 12:37:56.077721 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:56.077674    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 22 12:37:56.078281 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:56.077741    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 22 12:37:57.081326 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:57.081244    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:57.081834 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:57.081633    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:57.082394 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:57.082376    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:37:59.567861 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:59.567828    2565 patch_prober.go:28] interesting pod/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:41352->10.132.0.41:15021: read: connection reset by peer" start-of-body=
Jul 22 12:37:59.568288 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:59.567893    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:41352->10.132.0.41:15021: read: connection reset by peer"
Jul 22 12:37:59.655154 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:59.655125    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:59.756949 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:37:59.756856    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:38:00.374801 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.374765    2565 generic.go:356] "Generic (PLEG): container finished" podID="06f68167-db52-4110-9e81-a147cdc8082f" containerID="e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7" exitCode=0
Jul 22 12:38:00.374984 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.374823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerDied","Data":"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"}
Jul 22 12:38:00.375226 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.375205    2565 scope.go:117] "RemoveContainer" containerID="e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"
Jul 22 12:38:00.376221 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.376190    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:38:00.376298 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.376278    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:38:00.376343 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:00.376329    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:38:01.099018 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.098984    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:38:01.101093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.101075    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.104438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.104411    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5e8367\"/\"kube-root-ca.crt\""
Jul 22 12:38:01.104660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.104643    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5e8367\"/\"default-dockercfg-46xmf\""
Jul 22 12:38:01.104758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.104659    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5e8367\"/\"test-model-5e8367-kserve-self-signed-certs\""
Jul 22 12:38:01.104758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.104692    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5e8367\"/\"openshift-service-ca.crt\""
Jul 22 12:38:01.112132 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.112102    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:38:01.127200 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.127166    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:38:01.239546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.239546 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z79sb\" (UniqueName: \"kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.239770 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.239853 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.239945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239926    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.240011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.239988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.340924 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.340878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341116 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.340942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z79sb\" (UniqueName: \"kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341116 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341008    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341116 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341116 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341351 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341411 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341352    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.341923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.341601    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.345466 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.345439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.345675 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.345655    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.351315 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.351183    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z79sb\" (UniqueName: \"kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb\") pod \"test-model-5e8367-kserve-6bd6c54749-5nrv4\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") " pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.380678 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.380642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerStarted","Data":"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"}
Jul 22 12:38:01.410663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.410629    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:01.599002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.598977    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:38:01.601107 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:38:01.601076    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod709f8ee8_03b8_4336_8511_b309d36d4bba.slice/crio-f8f7ca806a78ebc61711c5240e8c0e36df10810b2aa142c72d107c63bf5b7e56 WatchSource:0}: Error finding container f8f7ca806a78ebc61711c5240e8c0e36df10810b2aa142c72d107c63bf5b7e56: Status 404 returned error can't find the container with id f8f7ca806a78ebc61711c5240e8c0e36df10810b2aa142c72d107c63bf5b7e56
Jul 22 12:38:01.668661 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.668629    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:38:01.668901 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.668878    2565 patch_prober.go:28] interesting pod/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 22 12:38:01.669005 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:01.668932    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 22 12:38:02.386644 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:02.386599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" event={"ID":"709f8ee8-03b8-4336-8511-b309d36d4bba","Type":"ContainerStarted","Data":"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"}
Jul 22 12:38:02.387040 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:02.386654    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" event={"ID":"709f8ee8-03b8-4336-8511-b309d36d4bba","Type":"ContainerStarted","Data":"f8f7ca806a78ebc61711c5240e8c0e36df10810b2aa142c72d107c63bf5b7e56"}
Jul 22 12:38:02.410598 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:02.410549    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" podStartSLOduration=1.410534873 podStartE2EDuration="1.410534873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:38:02.409012687 +0000 UTC m=+2202.832443740" watchObservedRunningTime="2026-07-22 12:38:02.410534873 +0000 UTC m=+2202.833965928"
Jul 22 12:38:02.668776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:02.668696    2565 patch_prober.go:28] interesting pod/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 22 12:38:02.668776 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:02.668757    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 22 12:38:03.669502 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:03.669460    2565 patch_prober.go:28] interesting pod/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 22 12:38:03.669917 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:03.669526    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 22 12:38:04.406449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:04.406414    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:38:04.669407 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:04.669319    2565 patch_prober.go:28] interesting pod/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 22 12:38:04.669407 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:04.669377    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 22 12:38:05.673022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:05.672989    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:38:05.673394 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:05.673263    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:38:05.673986 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:05.673970    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:38:06.868738 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:06.868696    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:38:06.971973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:06.971937    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:38:11.410921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:11.410881    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:11.411384 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:11.410968    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:11.423468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:11.423444    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:11.438422 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:11.438400    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:38:15.501148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:15.501108    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:38:20.325269 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:20.325239    2565 scope.go:117] "RemoveContainer" containerID="e0e4ac6374019098fe6c6aa3f50ec65b230088bdd37336bbe46f50d934a3cf9f"
Jul 22 12:38:30.733256 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:30.733217    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:38:35.796394 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:35.796357    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:38:58.520731 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:38:58.520698    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:14.276201 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:14.276163    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:39:14.358064 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:14.358027    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:15.363005 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:15.362971    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:16.127532 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:16.127486    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:16.231605 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:16.231564    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:19.298366 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:19.298329    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:39:26.062053 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:26.062017    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:29.295989 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.295918    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" podUID="709f8ee8-03b8-4336-8511-b309d36d4bba" containerName="main" containerID="cri-o://bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e" gracePeriod=45
Jul 22 12:39:29.550372 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.550313    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:39:29.676443 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676406    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676509    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676570    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676616    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676647    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z79sb\" (UniqueName: \"kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676678    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home\") pod \"709f8ee8-03b8-4336-8511-b309d36d4bba\" (UID: \"709f8ee8-03b8-4336-8511-b309d36d4bba\") "
Jul 22 12:39:29.676923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676875    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:29.676923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676909    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache" (OuterVolumeSpecName: "model-cache") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:29.677042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.676999    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.677042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.677018    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.677106 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.677031    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home" (OuterVolumeSpecName: "home") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:29.678933 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.678914    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb" (OuterVolumeSpecName: "kube-api-access-z79sb") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "kube-api-access-z79sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:29.679009 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.678993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm" (OuterVolumeSpecName: "dshm") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:29.679046 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.679017    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "709f8ee8-03b8-4336-8511-b309d36d4bba" (UID: "709f8ee8-03b8-4336-8511-b309d36d4bba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:29.778217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.778180    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.778217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.778207    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/709f8ee8-03b8-4336-8511-b309d36d4bba-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.778217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.778218    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z79sb\" (UniqueName: \"kubernetes.io/projected/709f8ee8-03b8-4336-8511-b309d36d4bba-kube-api-access-z79sb\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.778217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.778227    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/709f8ee8-03b8-4336-8511-b309d36d4bba-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:29.779996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.779971    2565 generic.go:356] "Generic (PLEG): container finished" podID="709f8ee8-03b8-4336-8511-b309d36d4bba" containerID="bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e" exitCode=0
Jul 22 12:39:29.780098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.780017    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"
Jul 22 12:39:29.780098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.780030    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" event={"ID":"709f8ee8-03b8-4336-8511-b309d36d4bba","Type":"ContainerDied","Data":"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"}
Jul 22 12:39:29.780098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.780063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4" event={"ID":"709f8ee8-03b8-4336-8511-b309d36d4bba","Type":"ContainerDied","Data":"f8f7ca806a78ebc61711c5240e8c0e36df10810b2aa142c72d107c63bf5b7e56"}
Jul 22 12:39:29.780098 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.780079    2565 scope.go:117] "RemoveContainer" containerID="bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"
Jul 22 12:39:29.789001 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.788982    2565 scope.go:117] "RemoveContainer" containerID="bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"
Jul 22 12:39:29.789248 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:39:29.789230    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e\": container with ID starting with bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e not found: ID does not exist" containerID="bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"
Jul 22 12:39:29.789302 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.789257    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e"} err="failed to get container status \"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e\": rpc error: code = NotFound desc = could not find container \"bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e\": container with ID starting with bf00e1460ac841fec833e727f801e2cc47c5eb278a6d3e8e612dac472312ad1e not found: ID does not exist"
Jul 22 12:39:29.801292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.801234    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:39:29.807536 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:29.807515    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5e8367/test-model-5e8367-kserve-6bd6c54749-5nrv4"]
Jul 22 12:39:30.135686 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:30.135608    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709f8ee8-03b8-4336-8511-b309d36d4bba" path="/var/lib/kubelet/pods/709f8ee8-03b8-4336-8511-b309d36d4bba/volumes"
Jul 22 12:39:30.541996 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:30.541962    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:30.962990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:30.962874    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:35.411915 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:35.411871    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:39:37.398081 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.398042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:37.498342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.498311    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:37.640197 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.640159    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:39:37.640430 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.640404    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" containerID="cri-o://3fc368122383a490e3e3d514f2d9bc26b04b6cf83a96a9faa5172a17f31937d5" gracePeriod=30
Jul 22 12:39:37.657732 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.657653    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:37.926673 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.926587    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:37.990289 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.990258    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"]
Jul 22 12:39:37.990574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.990562    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709f8ee8-03b8-4336-8511-b309d36d4bba" containerName="main"
Jul 22 12:39:37.990619 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.990576    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="709f8ee8-03b8-4336-8511-b309d36d4bba" containerName="main"
Jul 22 12:39:37.990666 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.990631    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="709f8ee8-03b8-4336-8511-b309d36d4bba" containerName="main"
Jul 22 12:39:37.994701 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.994683    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:37.996763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.996740    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:39:37.996878 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:37.996775    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:39:38.002040 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.001963    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"]
Jul 22 12:39:38.045857 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.045780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-07497a-czc72\" (UID: \"c97de25a-c8e0-4905-92b0-7fbc6acb0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:38.146705 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.146669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-07497a-czc72\" (UID: \"c97de25a-c8e0-4905-92b0-7fbc6acb0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:38.147277 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.147258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-07497a-czc72\" (UID: \"c97de25a-c8e0-4905-92b0-7fbc6acb0f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:38.305332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.305303    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:38.444131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.444006    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"]
Jul 22 12:39:38.447110 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:39:38.447086    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97de25a_c8e0_4905_92b0_7fbc6acb0f70.slice/crio-1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626 WatchSource:0}: Error finding container 1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626: Status 404 returned error can't find the container with id 1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626
Jul 22 12:39:38.566857 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.566747    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:38.664682 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.664644    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:38.822134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.822051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72" event={"ID":"c97de25a-c8e0-4905-92b0-7fbc6acb0f70","Type":"ContainerStarted","Data":"78e9dc9899dbdf460b808143b3093639c3e2335255667212b130780eaa93f0e9"}
Jul 22 12:39:38.822134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:38.822107    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72" event={"ID":"c97de25a-c8e0-4905-92b0-7fbc6acb0f70","Type":"ContainerStarted","Data":"1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626"}
Jul 22 12:39:39.828797 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:39.828764    2565 generic.go:356] "Generic (PLEG): container finished" podID="c97de25a-c8e0-4905-92b0-7fbc6acb0f70" containerID="78e9dc9899dbdf460b808143b3093639c3e2335255667212b130780eaa93f0e9" exitCode=0
Jul 22 12:39:39.829182 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:39.828855    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72" event={"ID":"c97de25a-c8e0-4905-92b0-7fbc6acb0f70","Type":"ContainerDied","Data":"78e9dc9899dbdf460b808143b3093639c3e2335255667212b130780eaa93f0e9"}
Jul 22 12:39:40.960585 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:40.960561    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:41.074002 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.073963    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca\") pod \"c97de25a-c8e0-4905-92b0-7fbc6acb0f70\" (UID: \"c97de25a-c8e0-4905-92b0-7fbc6acb0f70\") "
Jul 22 12:39:41.074367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.074337    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c97de25a-c8e0-4905-92b0-7fbc6acb0f70" (UID: "c97de25a-c8e0-4905-92b0-7fbc6acb0f70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:41.175077 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.174973    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c97de25a-c8e0-4905-92b0-7fbc6acb0f70-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:41.838937 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.838903    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72" event={"ID":"c97de25a-c8e0-4905-92b0-7fbc6acb0f70","Type":"ContainerDied","Data":"1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626"}
Jul 22 12:39:41.839108 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.838953    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e2339226f2289ce94a09ae0a8714bb6ad353ba6c69e45322fbff524cdd60626"
Jul 22 12:39:41.839108 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:41.838915    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"
Jul 22 12:39:42.083652 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.083616    2565 patch_prober.go:28] interesting pod/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:39:42.084104 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.083676    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:39:42.845687 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.845652    2565 generic.go:356] "Generic (PLEG): container finished" podID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerID="3fc368122383a490e3e3d514f2d9bc26b04b6cf83a96a9faa5172a17f31937d5" exitCode=0
Jul 22 12:39:42.845852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.845722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerDied","Data":"3fc368122383a490e3e3d514f2d9bc26b04b6cf83a96a9faa5172a17f31937d5"}
Jul 22 12:39:42.845852 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.845783    2565 scope.go:117] "RemoveContainer" containerID="d103103ae11da16ef0d896e4f1fcdc3ed78a8a163e3dab64c6b89ee0b64be613"
Jul 22 12:39:42.897056 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.897033    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:39:42.989908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989774    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.989908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989846    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.989908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.989908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989906    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989947    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh72b\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.989984    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990018    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990100    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert\") pod \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\" (UID: \"34e88a9f-1fa4-4251-a1f1-da913e8533fb\") "
Jul 22 12:39:42.990492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990287    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:42.990492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990309    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:42.990492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990403    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:42.990492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990422    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:42.990704 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990553    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:42.990704 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.990686    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:42.991138 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.991115    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:42.992161 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.992137    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:42.992279 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.992228    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:42.992518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.992492    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:39:42.992518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:42.992500    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b" (OuterVolumeSpecName: "kube-api-access-vh72b") pod "34e88a9f-1fa4-4251-a1f1-da913e8533fb" (UID: "34e88a9f-1fa4-4251-a1f1-da913e8533fb"). InnerVolumeSpecName "kube-api-access-vh72b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:43.091073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091037    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh72b\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-kube-api-access-vh72b\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091067    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091076    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091073 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091085    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091548 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091093    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091548 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091101    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.091548 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.091108    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e88a9f-1fa4-4251-a1f1-da913e8533fb-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:39:43.851107 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.851079    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"
Jul 22 12:39:43.851107 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.851098    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87" event={"ID":"34e88a9f-1fa4-4251-a1f1-da913e8533fb","Type":"ContainerDied","Data":"00ca83d4f8675a3fba0ba7fce2a7e9599419f838ccd9f372d1dff37f3c9020c3"}
Jul 22 12:39:43.851371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.851149    2565 scope.go:117] "RemoveContainer" containerID="3fc368122383a490e3e3d514f2d9bc26b04b6cf83a96a9faa5172a17f31937d5"
Jul 22 12:39:43.877993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.877967    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:39:43.880130 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:43.880105    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-5e8367-openshift-default-649ccc8865-qcm87"]
Jul 22 12:39:44.133994 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:44.133918    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" path="/var/lib/kubelet/pods/34e88a9f-1fa4-4251-a1f1-da913e8533fb/volumes"
Jul 22 12:39:48.568855 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:48.568817    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:51.502010 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:51.501974    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:39:53.095050 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:53.095020    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"]
Jul 22 12:39:53.097892 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:53.097848    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-07497a-czc72"]
Jul 22 12:39:53.103907 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:53.103880    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:53.489058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:53.488960    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:54.134176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:54.134139    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97de25a-c8e0-4905-92b0-7fbc6acb0f70" path="/var/lib/kubelet/pods/c97de25a-c8e0-4905-92b0-7fbc6acb0f70/volumes"
Jul 22 12:39:59.746959 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:59.746929    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:39:59.816959 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:39:59.816928    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:00.004816 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.004701    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:40:00.004968 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.004942    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy" containerID="cri-o://27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1" gracePeriod=30
Jul 22 12:40:00.034183 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.034143    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:00.444018 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.443982    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:00.619439 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.619403    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:00.966633 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:00.966593    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:01.261025 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:01.260947    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:03.958994 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:03.958959    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:04.039660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:04.039616    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:04.123367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:04.123331    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:04.394405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:04.394370    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:04.683991 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:04.683901    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:04.783796 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:04.783760    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:05.254362 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.254339    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:40:05.374709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374624    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374664    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374688    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374716    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntgff\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374747    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374828    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374870    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.374990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374901    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.375240 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.374986    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:05.375240 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375059    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:05.375240 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375116    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:40:05.375240 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data\") pod \"06f68167-db52-4110-9e81-a147cdc8082f\" (UID: \"06f68167-db52-4110-9e81-a147cdc8082f\") "
Jul 22 12:40:05.375240 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375151    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:05.375498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375408    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f68167-db52-4110-9e81-a147cdc8082f-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.375498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375427    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.375498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375440    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.375498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375454    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.376022 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.375994    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data" (OuterVolumeSpecName: "istio-data") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:05.377147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.377113    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:40:05.377147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.377123    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff" (OuterVolumeSpecName: "kube-api-access-ntgff") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "kube-api-access-ntgff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:40:05.377291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.377159    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token" (OuterVolumeSpecName: "istio-token") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:40:05.377291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.377196    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06f68167-db52-4110-9e81-a147cdc8082f" (UID: "06f68167-db52-4110-9e81-a147cdc8082f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:40:05.476580 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.476544    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f68167-db52-4110-9e81-a147cdc8082f-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.476580 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.476573    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.476580 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.476583    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f68167-db52-4110-9e81-a147cdc8082f-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.476850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.476591    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntgff\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-kube-api-access-ntgff\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.476850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.476601    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f68167-db52-4110-9e81-a147cdc8082f-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:05.953620 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.953582    2565 generic.go:356] "Generic (PLEG): container finished" podID="06f68167-db52-4110-9e81-a147cdc8082f" containerID="27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1" exitCode=0
Jul 22 12:40:05.953836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.953640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerDied","Data":"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"}
Jul 22 12:40:05.953836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.953656    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"
Jul 22 12:40:05.953836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.953707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9" event={"ID":"06f68167-db52-4110-9e81-a147cdc8082f","Type":"ContainerDied","Data":"439866fe411a107bd7944597eda4abcd6d84ba2b36c035bcd8008f546262b8c1"}
Jul 22 12:40:05.953836 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.953729    2565 scope.go:117] "RemoveContainer" containerID="27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"
Jul 22 12:40:05.962590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.962570    2565 scope.go:117] "RemoveContainer" containerID="e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"
Jul 22 12:40:05.970688 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.970666    2565 scope.go:117] "RemoveContainer" containerID="27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"
Jul 22 12:40:05.970968 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:40:05.970948    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1\": container with ID starting with 27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1 not found: ID does not exist" containerID="27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"
Jul 22 12:40:05.971031 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.970978    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1"} err="failed to get container status \"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1\": rpc error: code = NotFound desc = could not find container \"27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1\": container with ID starting with 27f03f5a316e80ebbe5b06f037aa3e646c75dbb19faeb0131cc8fce6a2d707c1 not found: ID does not exist"
Jul 22 12:40:05.971031 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.970996    2565 scope.go:117] "RemoveContainer" containerID="e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"
Jul 22 12:40:05.971219 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:40:05.971201    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7\": container with ID starting with e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7 not found: ID does not exist" containerID="e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"
Jul 22 12:40:05.971276 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.971229    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7"} err="failed to get container status \"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7\": rpc error: code = NotFound desc = could not find container \"e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7\": container with ID starting with e95346a6480e2005be63d32bf0aeff0c9f2a283aae985332360895e65c9bd5f7 not found: ID does not exist"
Jul 22 12:40:05.976709 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.976687    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:40:05.981542 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:05.981521    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-07497a-openshift-default-cd4dcb4d7-rmrp9"]
Jul 22 12:40:06.133559 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:06.133527    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f68167-db52-4110-9e81-a147cdc8082f" path="/var/lib/kubelet/pods/06f68167-db52-4110-9e81-a147cdc8082f/volumes"
Jul 22 12:40:08.460823 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:08.460772    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:08.925208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:08.925175    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:09.206680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206603    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:40:09.206922 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206910    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:09.206973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206924    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:09.206973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206933    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:09.206973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206939    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:09.206973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206952    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c97de25a-c8e0-4905-92b0-7fbc6acb0f70" containerName="revoke-keys"
Jul 22 12:40:09.206973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.206957    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97de25a-c8e0-4905-92b0-7fbc6acb0f70" containerName="revoke-keys"
Jul 22 12:40:09.207131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.207014    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:09.207131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.207021    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:09.207131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.207027    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:09.207131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.207033    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c97de25a-c8e0-4905-92b0-7fbc6acb0f70" containerName="revoke-keys"
Jul 22 12:40:09.207131 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.207038    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:09.211368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.211346    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.213427 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.213407    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6749ef-openshift-default-dockercfg-cdnvw\""
Jul 22 12:40:09.223342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.223313    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:40:09.267445 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.267407    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:09.309389 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309562    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309705    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.309740 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.309728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2p5m\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410738 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2p5m\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410871    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.410939 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.410977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411315 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411266    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411398 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411365    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411526 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411526 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.411646 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.411539    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.413271 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.413246    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.413494 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.413478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.420011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.419992    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.420011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.420002    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2p5m\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m\") pod \"e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") " pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.520866 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.520835    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:09.669403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.669378    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:40:09.671575 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:40:09.671546    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6320b22_4a10_4fc2_b260_7631e283e30d.slice/crio-c2eb457edabd563e7dea0a1cad84abb95ec1e8361aef3c2dec0e3f8b561a1d1d WatchSource:0}: Error finding container c2eb457edabd563e7dea0a1cad84abb95ec1e8361aef3c2dec0e3f8b561a1d1d: Status 404 returned error can't find the container with id c2eb457edabd563e7dea0a1cad84abb95ec1e8361aef3c2dec0e3f8b561a1d1d
Jul 22 12:40:09.673526 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.673491    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:09.673615 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.673571    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:09.673680 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.673616    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:09.676638 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.676609    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:09.796312 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.796223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:40:09.973408 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.973373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerStarted","Data":"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"}
Jul 22 12:40:09.973601 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.973423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerStarted","Data":"c2eb457edabd563e7dea0a1cad84abb95ec1e8361aef3c2dec0e3f8b561a1d1d"}
Jul 22 12:40:09.996579 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:09.996530    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podStartSLOduration=0.996513836 podStartE2EDuration="996.513836ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:40:09.995179164 +0000 UTC m=+2330.418610214" watchObservedRunningTime="2026-07-22 12:40:09.996513836 +0000 UTC m=+2330.419944891"
Jul 22 12:40:10.521671 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:10.521619    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:10.526614 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:10.526587    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:10.978217 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:10.978129    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:10.979048 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:10.979027    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:11.932964 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:11.932919    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:12.033758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:12.033723    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:12.112143 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:12.112112    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:12.441095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:12.441014    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:12.769175 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:12.769144    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:12.870095 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:12.870052    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:16.429292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:16.429248    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:20.351741 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:20.351616    2565 scope.go:117] "RemoveContainer" containerID="916787a2beea18ee2a5e166c037412f6c0a6a3edc8b6b3d9c75e1777627f5f3a"
Jul 22 12:40:24.445985 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.445957    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"]
Jul 22 12:40:24.448170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.446262    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:24.448170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.446274    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e88a9f-1fa4-4251-a1f1-da913e8533fb" containerName="istio-proxy"
Jul 22 12:40:24.448170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.446282    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:24.448170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.446288    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f68167-db52-4110-9e81-a147cdc8082f" containerName="istio-proxy"
Jul 22 12:40:24.449284 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.449268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:24.451624 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.451598    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:40:24.451758 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.451601    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:40:24.461495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.460980    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"]
Jul 22 12:40:24.544025 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.543994    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-968672-cg5wl\" (UID: \"1287637b-4304-4c84-97bf-d9b10a28e0dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:24.644907 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.644864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-968672-cg5wl\" (UID: \"1287637b-4304-4c84-97bf-d9b10a28e0dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:24.645544 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.645515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-968672-cg5wl\" (UID: \"1287637b-4304-4c84-97bf-d9b10a28e0dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:24.763272 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.763171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:24.889512 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:24.889479    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"]
Jul 22 12:40:24.892073 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:40:24.892048    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1287637b_4304_4c84_97bf_d9b10a28e0dc.slice/crio-b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af WatchSource:0}: Error finding container b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af: Status 404 returned error can't find the container with id b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af
Jul 22 12:40:25.018254 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.018209    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:25.042118 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.042094    2565 generic.go:356] "Generic (PLEG): container finished" podID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerID="313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1" exitCode=0
Jul 22 12:40:25.042244 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.042172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerDied","Data":"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"}
Jul 22 12:40:25.042590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.042573    2565 scope.go:117] "RemoveContainer" containerID="313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"
Jul 22 12:40:25.043503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.043479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl" event={"ID":"1287637b-4304-4c84-97bf-d9b10a28e0dc","Type":"ContainerStarted","Data":"b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af"}
Jul 22 12:40:25.043651 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.043624    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:25.043705 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.043691    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:25.043743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.043724    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:40:25.117428 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.117366    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:25.978137 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:25.978041    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:26.052484 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.052450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerStarted","Data":"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"}
Jul 22 12:40:26.056462 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.056127    2565 generic.go:356] "Generic (PLEG): container finished" podID="1287637b-4304-4c84-97bf-d9b10a28e0dc" containerID="ef0a9ce60ed1f839a9e045e77bc56a70564983094cc8e33380ea08daecfa7f5f" exitCode=0
Jul 22 12:40:26.056462 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.056172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl" event={"ID":"1287637b-4304-4c84-97bf-d9b10a28e0dc","Type":"ContainerDied","Data":"ef0a9ce60ed1f839a9e045e77bc56a70564983094cc8e33380ea08daecfa7f5f"}
Jul 22 12:40:26.521435 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.521398    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:26.522480 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.522455    2565 patch_prober.go:28] interesting pod/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:40:26.522595 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.522513    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 22 12:40:26.812810 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:26.812703    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:40:27.211728 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.211703    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:27.263042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.263010    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca\") pod \"1287637b-4304-4c84-97bf-d9b10a28e0dc\" (UID: \"1287637b-4304-4c84-97bf-d9b10a28e0dc\") "
Jul 22 12:40:27.263354 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.263333    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1287637b-4304-4c84-97bf-d9b10a28e0dc" (UID: "1287637b-4304-4c84-97bf-d9b10a28e0dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:40:27.363687 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.363652    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1287637b-4304-4c84-97bf-d9b10a28e0dc-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:27.521496 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.521462    2565 patch_prober.go:28] interesting pod/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:40:27.521658 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:27.521520    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 22 12:40:28.066800 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:28.066755    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"
Jul 22 12:40:28.066969 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:28.066813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl" event={"ID":"1287637b-4304-4c84-97bf-d9b10a28e0dc","Type":"ContainerDied","Data":"b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af"}
Jul 22 12:40:28.066969 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:28.066856    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b07bda2f83748e8bbd83a74a4d2c50ec0709c3a15bf6b20f19f2d4d8c480f5af"
Jul 22 12:40:28.521862 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:28.521826    2565 patch_prober.go:28] interesting pod/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:40:28.522309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:28.521904    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 22 12:40:29.521734 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:29.521696    2565 patch_prober.go:28] interesting pod/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 22 12:40:29.521910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:29.521750    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 22 12:40:30.525711 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:30.525679    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:30.527853 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:30.527347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:30.531676 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:30.531646    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:40:35.005194 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:35.005161    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:35.105659 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:35.105624    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:35.326920 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:35.326885    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:35.429255 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:35.429219    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:40.299439 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:40.299409    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:44.200529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:44.200492    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:40:44.530387 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:44.530346    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:44.545887 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:44.545855    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"]
Jul 22 12:40:44.552501 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:44.552477    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-968672-cg5wl"]
Jul 22 12:40:44.929080 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:44.928989    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:46.133406 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:46.133372    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1287637b-4304-4c84-97bf-d9b10a28e0dc" path="/var/lib/kubelet/pods/1287637b-4304-4c84-97bf-d9b10a28e0dc/volumes"
Jul 22 12:40:51.431643 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:51.431605    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:51.531742 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:51.531704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:51.723245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:51.723157    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:51.992434 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:51.992330    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:40:52.049200 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.049159    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"]
Jul 22 12:40:52.049547 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.049529    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1287637b-4304-4c84-97bf-d9b10a28e0dc" containerName="revoke-keys"
Jul 22 12:40:52.049630 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.049549    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1287637b-4304-4c84-97bf-d9b10a28e0dc" containerName="revoke-keys"
Jul 22 12:40:52.049684 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.049646    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1287637b-4304-4c84-97bf-d9b10a28e0dc" containerName="revoke-keys"
Jul 22 12:40:52.053633 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.053611    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:52.057606 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.057585    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:40:52.057716 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.057622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:40:52.065864 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.065833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"]
Jul 22 12:40:52.171772 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.171736    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g\" (UID: \"46a81d8e-5a3d-4eba-acfc-afca57306796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:52.272129 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.272097    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g\" (UID: \"46a81d8e-5a3d-4eba-acfc-afca57306796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:52.272841 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.272822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g\" (UID: \"46a81d8e-5a3d-4eba-acfc-afca57306796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:52.363945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.363904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:52.498317 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:52.498293    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"]
Jul 22 12:40:52.500200 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:40:52.500173    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a81d8e_5a3d_4eba_acfc_afca57306796.slice/crio-4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871 WatchSource:0}: Error finding container 4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871: Status 404 returned error can't find the container with id 4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871
Jul 22 12:40:53.177634 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:53.177605    2565 generic.go:356] "Generic (PLEG): container finished" podID="46a81d8e-5a3d-4eba-acfc-afca57306796" containerID="0a1041b6c122d38c06f3484887c4255e0cbf3464ca64651d706630d9a18f69cf" exitCode=0
Jul 22 12:40:53.177771 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:53.177693    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g" event={"ID":"46a81d8e-5a3d-4eba-acfc-afca57306796","Type":"ContainerDied","Data":"0a1041b6c122d38c06f3484887c4255e0cbf3464ca64651d706630d9a18f69cf"}
Jul 22 12:40:53.177771 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:53.177756    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g" event={"ID":"46a81d8e-5a3d-4eba-acfc-afca57306796","Type":"ContainerStarted","Data":"4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871"}
Jul 22 12:40:54.308714 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:54.308688    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:54.391241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:54.391203    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca\") pod \"46a81d8e-5a3d-4eba-acfc-afca57306796\" (UID: \"46a81d8e-5a3d-4eba-acfc-afca57306796\") "
Jul 22 12:40:54.391591 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:54.391564    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46a81d8e-5a3d-4eba-acfc-afca57306796" (UID: "46a81d8e-5a3d-4eba-acfc-afca57306796"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:40:54.492097 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:54.492014    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a81d8e-5a3d-4eba-acfc-afca57306796-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:40:55.187735 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:55.187706    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"
Jul 22 12:40:55.187925 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:55.187736    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g" event={"ID":"46a81d8e-5a3d-4eba-acfc-afca57306796","Type":"ContainerDied","Data":"4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871"}
Jul 22 12:40:55.187925 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:40:55.187777    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ec79648baf500c8dc848ca330bb6fd6e1a526efac2ac9fe25c0b003aeb91871"
Jul 22 12:41:07.536582 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:07.536547    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:12.153785 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:12.153746    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"]
Jul 22 12:41:12.162118 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:12.162084    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6749ef-kzk9g"]
Jul 22 12:41:12.188065 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:12.188037    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:12.502764 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:12.502668    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:41:12.541058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:12.541013    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:14.133660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:14.133624    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46a81d8e-5a3d-4eba-acfc-afca57306796" path="/var/lib/kubelet/pods/46a81d8e-5a3d-4eba-acfc-afca57306796/volumes"
Jul 22 12:41:19.155500 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.155467    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:19.226303 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.226252    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:19.424851 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.424754    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:41:19.425014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.424965    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy" containerID="cri-o://d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e" gracePeriod=30
Jul 22 12:41:19.449752 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.449711    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:19.695157 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:19.695064    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:20.037767 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.037731    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:20.325595 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.325499    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:41:20.326058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.325945    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46a81d8e-5a3d-4eba-acfc-afca57306796" containerName="revoke-keys"
Jul 22 12:41:20.326058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.325964    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="46a81d8e-5a3d-4eba-acfc-afca57306796" containerName="revoke-keys"
Jul 22 12:41:20.326178 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.326082    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="46a81d8e-5a3d-4eba-acfc-afca57306796" containerName="revoke-keys"
Jul 22 12:41:20.329219 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.329198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.331495 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.331477    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a12988-openshift-default-dockercfg-llpmz\""
Jul 22 12:41:20.342739 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.342712    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:41:20.352089 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.352061    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:20.418667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418727    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418752    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.418903 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418887    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.419149 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.418957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjwtt\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.419149 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.419000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.423247 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.423197    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:20.519743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519707    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjwtt\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.519963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.519942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.520409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.520220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.520409 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.520363    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.520519 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.520406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.520519 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.520476    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.520816 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.520767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.522292 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.522272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.522405 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.522387    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.529257 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.529233    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.529343 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.529332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjwtt\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt\") pod \"e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") " pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.640934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.640840    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:20.779231 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.779202    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:41:20.781362 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:41:20.781332    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98c472c_5615_4e79_b00f_3f979d0b43ba.slice/crio-b3d6c7a5a428e357a66333f5deb2502ff8cb5fca1da9c16526632451b5643583 WatchSource:0}: Error finding container b3d6c7a5a428e357a66333f5deb2502ff8cb5fca1da9c16526632451b5643583: Status 404 returned error can't find the container with id b3d6c7a5a428e357a66333f5deb2502ff8cb5fca1da9c16526632451b5643583
Jul 22 12:41:20.783418 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.783383    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:20.783506 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.783450    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:20.783506 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:20.783477    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:21.309166 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:21.309129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerStarted","Data":"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"}
Jul 22 12:41:21.309349 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:21.309193    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerStarted","Data":"b3d6c7a5a428e357a66333f5deb2502ff8cb5fca1da9c16526632451b5643583"}
Jul 22 12:41:21.331272 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:21.331206    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podStartSLOduration=1.331187232 podStartE2EDuration="1.331187232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:41:21.329679323 +0000 UTC m=+2401.753110392" watchObservedRunningTime="2026-07-22 12:41:21.331187232 +0000 UTC m=+2401.754618289"
Jul 22 12:41:21.641492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:21.641391    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:21.646395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:21.646367    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:22.314383 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:22.314346    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:22.315378 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:22.315358    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:23.251840 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:23.251782    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:23.357406 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:23.357359    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:23.530473 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:23.530440    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:23.777832 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:23.777777    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:24.269659 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.269620    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:24.368942 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.368905    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:24.684756 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.684729    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:41:24.756629 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.756591    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.756629 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.756629    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2p5m\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.756919 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.756664    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.757205 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.757176    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.757342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.757244    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.757342 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.757275    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.757997 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.757315    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.758208 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758181    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:41:24.758331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758313    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.758472 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758456    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket\") pod \"d6320b22-4a10-4fc2-b260-7631e283e30d\" (UID: \"d6320b22-4a10-4fc2-b260-7631e283e30d\") "
Jul 22 12:41:24.758669 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758361    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:41:24.758669 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758505    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:41:24.758669 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758605    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:41:24.758947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.758922    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:41:24.759191 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759165    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.759191 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759191    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.759296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759204    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6320b22-4a10-4fc2-b260-7631e283e30d-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.759296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759215    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.759296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759228    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.759296 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759248    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m" (OuterVolumeSpecName: "kube-api-access-p2p5m") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "kube-api-access-p2p5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:41:24.759467 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759390    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:41:24.759825 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.759778    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:41:24.760294 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.760273    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6320b22-4a10-4fc2-b260-7631e283e30d" (UID: "d6320b22-4a10-4fc2-b260-7631e283e30d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:41:24.860093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.859993    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.860093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.860037    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.860093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.860047    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6320b22-4a10-4fc2-b260-7631e283e30d-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:24.860093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:24.860056    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2p5m\" (UniqueName: \"kubernetes.io/projected/d6320b22-4a10-4fc2-b260-7631e283e30d-kube-api-access-p2p5m\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:41:25.329635 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.329600    2565 generic.go:356] "Generic (PLEG): container finished" podID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerID="d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e" exitCode=0
Jul 22 12:41:25.330215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.329644    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"
Jul 22 12:41:25.330215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.329685    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerDied","Data":"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"}
Jul 22 12:41:25.330215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.329749    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr" event={"ID":"d6320b22-4a10-4fc2-b260-7631e283e30d","Type":"ContainerDied","Data":"c2eb457edabd563e7dea0a1cad84abb95ec1e8361aef3c2dec0e3f8b561a1d1d"}
Jul 22 12:41:25.330215 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.329808    2565 scope.go:117] "RemoveContainer" containerID="d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"
Jul 22 12:41:25.339367 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.339352    2565 scope.go:117] "RemoveContainer" containerID="313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"
Jul 22 12:41:25.347167 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.347150    2565 scope.go:117] "RemoveContainer" containerID="d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"
Jul 22 12:41:25.347415 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:41:25.347397    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e\": container with ID starting with d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e not found: ID does not exist" containerID="d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"
Jul 22 12:41:25.347461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.347423    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e"} err="failed to get container status \"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e\": rpc error: code = NotFound desc = could not find container \"d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e\": container with ID starting with d0ab4c41f8c7bd0b7d3db331a249800fda3d525e6d80861ac5ce9d15fc778b7e not found: ID does not exist"
Jul 22 12:41:25.347461 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.347440    2565 scope.go:117] "RemoveContainer" containerID="313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"
Jul 22 12:41:25.347627 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:41:25.347609    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1\": container with ID starting with 313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1 not found: ID does not exist" containerID="313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"
Jul 22 12:41:25.347667 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.347631    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1"} err="failed to get container status \"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1\": rpc error: code = NotFound desc = could not find container \"313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1\": container with ID starting with 313dd272554c8f45c909e75724ec7a6a11fdc8b035742b2ca1147681efc417c1 not found: ID does not exist"
Jul 22 12:41:25.352870 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.352846    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:41:25.360485 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:25.360463    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6749ef-openshift-default-6c786d96c5-ffdwr"]
Jul 22 12:41:26.133659 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:26.133622    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" path="/var/lib/kubelet/pods/d6320b22-4a10-4fc2-b260-7631e283e30d/volumes"
Jul 22 12:41:27.812122 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:27.812091    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:28.162994 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.162897    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:28.421707 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.421622    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:41:28.421988 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.421974    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.422039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.421990    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.422039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.422001    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.422039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.422009    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.422133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.422070    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.422133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.422079    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6320b22-4a10-4fc2-b260-7631e283e30d" containerName="istio-proxy"
Jul 22 12:41:28.425037 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.425017    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.427211 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.427190    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-980f27-openshift-default-dockercfg-6xxr9\""
Jul 22 12:41:28.437615 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.437591    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:41:28.465897 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.465859    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:28.489340 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8rxm\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489503 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489482    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.489648 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.489618    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.590706 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590672    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.590928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.590928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.590928 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591094 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591094 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.590991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591094 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8rxm\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591094 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591072    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591121    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591160    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591216    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591471 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591471 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591435    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.591711 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.591687    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.593147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.593130    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.593523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.593505    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.600525 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.600500    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.600932 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.600914    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8rxm\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm\") pod \"e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") " pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.735600 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.735514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:28.759952 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.759921    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:28.873652 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.873626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:41:28.876229 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:41:28.876203    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaead397_d760_4888_ad8d_53ea330c4157.slice/crio-0845575dc6c70d27ffac6994a332c06cd769639b348bd9ec759b15f9b95071c7 WatchSource:0}: Error finding container 0845575dc6c70d27ffac6994a332c06cd769639b348bd9ec759b15f9b95071c7: Status 404 returned error can't find the container with id 0845575dc6c70d27ffac6994a332c06cd769639b348bd9ec759b15f9b95071c7
Jul 22 12:41:28.878264 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.878232    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:28.878371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.878311    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:28.878371 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:28.878361    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:29.350451 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:29.350414    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerStarted","Data":"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"}
Jul 22 12:41:29.350660 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:29.350476    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerStarted","Data":"0845575dc6c70d27ffac6994a332c06cd769639b348bd9ec759b15f9b95071c7"}
Jul 22 12:41:29.378948 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:29.378895    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podStartSLOduration=1.378878707 podStartE2EDuration="1.378878707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:41:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:41:29.377202629 +0000 UTC m=+2409.800633699" watchObservedRunningTime="2026-07-22 12:41:29.378878707 +0000 UTC m=+2409.802309762"
Jul 22 12:41:29.736013 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:29.735913    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:29.741520 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:29.741493    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:30.354871 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:30.354835    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:30.355817 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:30.355781    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:31.202242 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:31.202201    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:41:31.319244 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:31.319168    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:31.416443 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:31.416409    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:31.882926 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:31.882891    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:31.983591 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:31.983551    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:32.348715 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:32.348680    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:32.483308 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:32.483252    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:32.834003 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:32.833967    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:32.935603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:32.935560    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:35.826594 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:35.826558    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:38.357016 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:38.356968    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 22 12:41:38.357434 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:38.357045    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 22 12:41:40.401946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.401913    2565 generic.go:356] "Generic (PLEG): container finished" podID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerID="ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9" exitCode=0
Jul 22 12:41:40.402386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.401985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerDied","Data":"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"}
Jul 22 12:41:40.402386 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.402373    2565 scope.go:117] "RemoveContainer" containerID="ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"
Jul 22 12:41:40.403427 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.403399    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:40.403498 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.403466    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:40.403536 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:40.403498    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:41.408493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:41.408454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerStarted","Data":"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"}
Jul 22 12:41:41.641780 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:41.641738    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:41.642286 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:41.642254    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 12:41:41.642401 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:41.642343    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 12:41:42.218025 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:42.217990    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:41:42.641473 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:42.641428    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 12:41:42.642111 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:42.642079    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 12:41:43.641663 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:43.641623    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 12:41:43.642206 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:43.641697    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 12:41:44.641902 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:44.641852    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 12:41:44.642410 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:44.641939    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 12:41:44.982291 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:44.982200    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:45.083020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.082987    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:45.644598 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.644564    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:45.645086 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.644977    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:45.645550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.645531    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:41:45.820538 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.820189    2565 patch_prober.go:28] interesting pod/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:35350->10.132.0.48:15021: read: connection reset by peer" start-of-body=
Jul 22 12:41:45.820538 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:45.820274    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": read tcp 10.132.0.2:35350->10.132.0.48:15021: read: connection reset by peer"
Jul 22 12:41:46.432899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.432860    2565 generic.go:356] "Generic (PLEG): container finished" podID="aaead397-d760-4888-ad8d-53ea330c4157" containerID="3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca" exitCode=0
Jul 22 12:41:46.433104 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.432937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerDied","Data":"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"}
Jul 22 12:41:46.433481 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.433459    2565 scope.go:117] "RemoveContainer" containerID="3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"
Jul 22 12:41:46.434449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.434422    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:46.434518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.434487    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:46.434558 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:46.434519    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 12:41:47.443088 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:47.443045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerStarted","Data":"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"}
Jul 22 12:41:47.737148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:47.736997    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:47.737463 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:47.737437    2565 patch_prober.go:28] interesting pod/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 22 12:41:47.737632 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:47.737606    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 22 12:41:48.736327 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:48.736282    2565 patch_prober.go:28] interesting pod/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 22 12:41:48.736881 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:48.736356    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 22 12:41:49.736615 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:49.736574    2565 patch_prober.go:28] interesting pod/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 22 12:41:49.737239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:49.737208    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 22 12:41:50.736526 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:50.736461    2565 patch_prober.go:28] interesting pod/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 22 12:41:50.736750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:50.736573    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 22 12:41:51.739430 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:51.739399    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:51.739950 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:51.739783    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:51.740375 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:51.740356    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:41:53.650057 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:53.650018    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:53.750510 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:53.750473    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:54.197761 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.197722    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:41:54.202502 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.202474    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.206176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.206148    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"kube-root-ca.crt\""
Jul 22 12:41:54.206309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.206148    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"openshift-service-ca.crt\""
Jul 22 12:41:54.206309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.206152    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"auth-test-model-a12988-kserve-self-signed-certs\""
Jul 22 12:41:54.206309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.206208    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"default-dockercfg-b5tsm\""
Jul 22 12:41:54.211727 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.211700    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:41:54.224363 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.224332    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:41:54.318574 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.318775 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.318775 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318639    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr4jm\" (UniqueName: \"kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.318775 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.318775 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318735    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.318990 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.318778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.419956 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.419907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.419989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420026    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr4jm\" (UniqueName: \"kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420067    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420169 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420517 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420493    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420517 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.420628 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.420561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.422336 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.422316    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.422726 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.422705    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.429440 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.429417    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr4jm\" (UniqueName: \"kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm\") pod \"auth-test-model-a12988-kserve-5b59dd66ff-dr6tx\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") " pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.513825 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.513716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:41:54.646047 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:54.646013    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:41:54.649058 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:41:54.649031    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69063bee_9030_48f4_85a5_63a070242e27.slice/crio-6d57031883992131bb95270a0256c3113919e094c7cf3a5b5ef4262851646ed8 WatchSource:0}: Error finding container 6d57031883992131bb95270a0256c3113919e094c7cf3a5b5ef4262851646ed8: Status 404 returned error can't find the container with id 6d57031883992131bb95270a0256c3113919e094c7cf3a5b5ef4262851646ed8
Jul 22 12:41:55.483965 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:55.483929    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" event={"ID":"69063bee-9030-48f4-85a5-63a070242e27","Type":"ContainerStarted","Data":"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"}
Jul 22 12:41:55.484353 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:55.483975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" event={"ID":"69063bee-9030-48f4-85a5-63a070242e27","Type":"ContainerStarted","Data":"6d57031883992131bb95270a0256c3113919e094c7cf3a5b5ef4262851646ed8"}
Jul 22 12:41:55.503186 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:55.503141    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" podStartSLOduration=1.503124502 podStartE2EDuration="1.503124502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:41:55.502343107 +0000 UTC m=+2435.925774172" watchObservedRunningTime="2026-07-22 12:41:55.503124502 +0000 UTC m=+2435.926555556"
Jul 22 12:41:57.497239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:41:57.497205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:42:04.514725 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:04.514681    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:42:04.515154 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:04.514741    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:42:04.527889 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:04.527864    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:42:04.543894 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:04.543865    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:42:20.392603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:20.392570    2565 scope.go:117] "RemoveContainer" containerID="d6c243921ac0e2c413a97b9f217c1e14fee1d408bd5b3fb76dd53246d2022414"
Jul 22 12:42:26.438611 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:26.438570    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:42:28.354945 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:28.354910    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:42:31.899741 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:31.899707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:42:47.808750 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:42:47.808714    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:43:05.808178 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:43:05.808143    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:43:09.132406 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:43:09.132369    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:43:22.103473 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:43:22.103387    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:43:40.404669 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:43:40.404632    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:43:58.810585 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:43:58.810551    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:44:17.801084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:17.801048    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:44:23.538264 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:23.538224    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:44:23.604107 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:23.604073    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:44:31.817561 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:31.817527    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:44:31.847341 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:31.847308    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:44:33.020816 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:33.020755    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:44:36.107947 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.107913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:44:36.506816 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.506721    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:44:36.509995 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.509979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.512014 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.511991    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"sub-test-model-a12988-kserve-self-signed-certs\""
Jul 22 12:44:36.519592 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.519566    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:44:36.527707 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sft4v\" (UniqueName: \"kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.527867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527727    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.527867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.527867 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.528011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.528011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.527944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.530833 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.530809    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:44:36.628855 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.628817    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.628866    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.628918    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sft4v\" (UniqueName: \"kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.628948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.628986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.629023    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629347 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.629290    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.629357    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.629430 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.629399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.631449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.631417    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.631449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.631429    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.638096 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.638071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sft4v\" (UniqueName: \"kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v\") pod \"sub-test-model-a12988-kserve-5689fc6b77-kvffv\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") " pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.819608 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.819567    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:36.967011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:36.966983    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:44:36.969685 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:44:36.969660    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b899067_2a30_4f2e_b442_472946e4ea0b.slice/crio-5b2ef2ce620fab2333c08bb0c376abe694f30ce599d3f7abc829a99ff694abe5 WatchSource:0}: Error finding container 5b2ef2ce620fab2333c08bb0c376abe694f30ce599d3f7abc829a99ff694abe5: Status 404 returned error can't find the container with id 5b2ef2ce620fab2333c08bb0c376abe694f30ce599d3f7abc829a99ff694abe5
Jul 22 12:44:37.242113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:37.242017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" event={"ID":"4b899067-2a30-4f2e-b442-472946e4ea0b","Type":"ContainerStarted","Data":"b59138d5b4c1ec3ac7aac8e6fc39ea640d853a3bea1e2f307d1e2670d0dea889"}
Jul 22 12:44:37.242113 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:37.242056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" event={"ID":"4b899067-2a30-4f2e-b442-472946e4ea0b","Type":"ContainerStarted","Data":"5b2ef2ce620fab2333c08bb0c376abe694f30ce599d3f7abc829a99ff694abe5"}
Jul 22 12:44:37.266018 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:37.265966    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" podStartSLOduration=1.265951563 podStartE2EDuration="1.265951563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:37.263951544 +0000 UTC m=+2597.687382598" watchObservedRunningTime="2026-07-22 12:44:37.265951563 +0000 UTC m=+2597.689382617"
Jul 22 12:44:46.820054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:46.819955    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:46.820054 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:46.820011    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:46.833058 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:46.833030    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:46.836956 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:46.836919    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" podUID="69063bee-9030-48f4-85a5-63a070242e27" containerName="main" containerID="cri-o://4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954" gracePeriod=45
Jul 22 12:44:47.089895 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.089869    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:44:47.112885 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.112862    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.112899    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.112926    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.112953    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr4jm\" (UniqueName: \"kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.112981    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113039 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.113028    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir\") pod \"69063bee-9030-48f4-85a5-63a070242e27\" (UID: \"69063bee-9030-48f4-85a5-63a070242e27\") "
Jul 22 12:44:47.113306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.113205    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache" (OuterVolumeSpecName: "model-cache") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:47.113306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.113233    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home" (OuterVolumeSpecName: "home") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:47.113306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.113294    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.113411 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.113339    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:47.115176 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.115153    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm" (OuterVolumeSpecName: "dshm") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:47.115265 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.115154    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:44:47.115265 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.115226    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm" (OuterVolumeSpecName: "kube-api-access-kr4jm") pod "69063bee-9030-48f4-85a5-63a070242e27" (UID: "69063bee-9030-48f4-85a5-63a070242e27"). InnerVolumeSpecName "kube-api-access-kr4jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:44:47.214147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.214106    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69063bee-9030-48f4-85a5-63a070242e27-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.214147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.214139    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.214147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.214148    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr4jm\" (UniqueName: \"kubernetes.io/projected/69063bee-9030-48f4-85a5-63a070242e27-kube-api-access-kr4jm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.214147 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.214156    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.214427 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.214168    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69063bee-9030-48f4-85a5-63a070242e27-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:44:47.288352 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.288318    2565 generic.go:356] "Generic (PLEG): container finished" podID="69063bee-9030-48f4-85a5-63a070242e27" containerID="4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954" exitCode=0
Jul 22 12:44:47.288518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.288362    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"
Jul 22 12:44:47.288518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.288403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" event={"ID":"69063bee-9030-48f4-85a5-63a070242e27","Type":"ContainerDied","Data":"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"}
Jul 22 12:44:47.288518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.288449    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx" event={"ID":"69063bee-9030-48f4-85a5-63a070242e27","Type":"ContainerDied","Data":"6d57031883992131bb95270a0256c3113919e094c7cf3a5b5ef4262851646ed8"}
Jul 22 12:44:47.288518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.288471    2565 scope.go:117] "RemoveContainer" containerID="4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"
Jul 22 12:44:47.297595 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.297561    2565 scope.go:117] "RemoveContainer" containerID="4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"
Jul 22 12:44:47.297887 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:44:47.297868    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954\": container with ID starting with 4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954 not found: ID does not exist" containerID="4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"
Jul 22 12:44:47.297955 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.297896    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954"} err="failed to get container status \"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954\": rpc error: code = NotFound desc = could not find container \"4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954\": container with ID starting with 4990ecad95b39da845ddc14698bfd72e6b69b134c152ac32a727e94976ea0954 not found: ID does not exist"
Jul 22 12:44:47.300480 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.300463    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:44:47.314815 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.314768    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:44:47.326212 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:47.324768    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/auth-test-model-a12988-kserve-5b59dd66ff-dr6tx"]
Jul 22 12:44:48.135327 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:48.135292    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69063bee-9030-48f4-85a5-63a070242e27" path="/var/lib/kubelet/pods/69063bee-9030-48f4-85a5-63a070242e27/volumes"
Jul 22 12:44:50.293166 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:44:50.293134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:45:09.784681 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:09.784631    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:45:13.496534 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:13.496495    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:45:30.102585 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:30.102546    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:45:48.305751 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:48.305719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:45:54.322902 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:54.322845    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:45:54.422350 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:54.422314    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:45:55.452971 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:55.452938    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:45:55.487449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:55.487417    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:45:56.308749 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:45:56.308718    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:02.398744 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.398707    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:46:02.399260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.399223    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69063bee-9030-48f4-85a5-63a070242e27" containerName="main"
Jul 22 12:46:02.399260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.399243    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69063bee-9030-48f4-85a5-63a070242e27" containerName="main"
Jul 22 12:46:02.399376 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.399336    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="69063bee-9030-48f4-85a5-63a070242e27" containerName="main"
Jul 22 12:46:02.402223 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.402201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.404170 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.404145    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a12988\"/\"rate-test-model-a12988-kserve-self-signed-certs\""
Jul 22 12:46:02.411723 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.411699    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:46:02.435906 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.435865    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:02.458559 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.458719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.458719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458648    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlnb\" (UniqueName: \"kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.458934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458708    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.458934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458762    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.458934 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.458820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.559756 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.559720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.559982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.559800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlnb\" (UniqueName: \"kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.559982 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.559849    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.559969    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.560020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560105 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.560071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560265 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.560201    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560309 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.560277    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.560375 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.560358    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.562202 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.562170    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.562490 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.562473    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.572148 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.572125    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlnb\" (UniqueName: \"kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb\") pod \"rate-test-model-a12988-kserve-6696dfc6cf-wccf9\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") " pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.711974 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.711888    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:02.851559 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:02.851498    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:46:02.853743 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:46:02.853716    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1b7b3c7_2c40_457a_8524_ef4deb73b1d3.slice/crio-db7c272238c8d2fa654e514b46f81a55294da6f89fc4d943c6f07233d1786c7d WatchSource:0}: Error finding container db7c272238c8d2fa654e514b46f81a55294da6f89fc4d943c6f07233d1786c7d: Status 404 returned error can't find the container with id db7c272238c8d2fa654e514b46f81a55294da6f89fc4d943c6f07233d1786c7d
Jul 22 12:46:03.650226 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:03.650193    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" event={"ID":"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3","Type":"ContainerStarted","Data":"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"}
Jul 22 12:46:03.650226 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:03.650233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" event={"ID":"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3","Type":"ContainerStarted","Data":"db7c272238c8d2fa654e514b46f81a55294da6f89fc4d943c6f07233d1786c7d"}
Jul 22 12:46:03.670412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:03.670364    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" podStartSLOduration=1.670347326 podStartE2EDuration="1.670347326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:03.669970197 +0000 UTC m=+2684.093401265" watchObservedRunningTime="2026-07-22 12:46:03.670347326 +0000 UTC m=+2684.093778380"
Jul 22 12:46:05.915610 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:05.915567    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:46:10.472647 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.472598    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" podUID="4b899067-2a30-4f2e-b442-472946e4ea0b" containerName="main" containerID="cri-o://b59138d5b4c1ec3ac7aac8e6fc39ea640d853a3bea1e2f307d1e2670d0dea889" gracePeriod=45
Jul 22 12:46:10.683000 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.682963    2565 generic.go:356] "Generic (PLEG): container finished" podID="4b899067-2a30-4f2e-b442-472946e4ea0b" containerID="b59138d5b4c1ec3ac7aac8e6fc39ea640d853a3bea1e2f307d1e2670d0dea889" exitCode=0
Jul 22 12:46:10.683000 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.682988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" event={"ID":"4b899067-2a30-4f2e-b442-472946e4ea0b","Type":"ContainerDied","Data":"b59138d5b4c1ec3ac7aac8e6fc39ea640d853a3bea1e2f307d1e2670d0dea889"}
Jul 22 12:46:10.722021 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.721997    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828398    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828497    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828538    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sft4v\" (UniqueName: \"kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828568    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828609    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.828658    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir\") pod \"4b899067-2a30-4f2e-b442-472946e4ea0b\" (UID: \"4b899067-2a30-4f2e-b442-472946e4ea0b\") "
Jul 22 12:46:10.829695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.829379    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:10.832725 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.832436    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home" (OuterVolumeSpecName: "home") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:10.833316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.833284    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:46:10.833679 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.833649    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:10.835459 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.835429    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v" (OuterVolumeSpecName: "kube-api-access-sft4v") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "kube-api-access-sft4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:46:10.835686 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.835653    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm" (OuterVolumeSpecName: "dshm") pod "4b899067-2a30-4f2e-b442-472946e4ea0b" (UID: "4b899067-2a30-4f2e-b442-472946e4ea0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:10.929838 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929777    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:10.929838 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929835    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sft4v\" (UniqueName: \"kubernetes.io/projected/4b899067-2a30-4f2e-b442-472946e4ea0b-kube-api-access-sft4v\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:10.929838 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929846    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:10.930075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929856    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:10.930075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929866    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b899067-2a30-4f2e-b442-472946e4ea0b-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:10.930075 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:10.929874    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b899067-2a30-4f2e-b442-472946e4ea0b-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:46:11.689256 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:11.689229    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"
Jul 22 12:46:11.689256 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:11.689229    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv" event={"ID":"4b899067-2a30-4f2e-b442-472946e4ea0b","Type":"ContainerDied","Data":"5b2ef2ce620fab2333c08bb0c376abe694f30ce599d3f7abc829a99ff694abe5"}
Jul 22 12:46:11.689751 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:11.689284    2565 scope.go:117] "RemoveContainer" containerID="b59138d5b4c1ec3ac7aac8e6fc39ea640d853a3bea1e2f307d1e2670d0dea889"
Jul 22 12:46:11.711743 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:11.711717    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:46:11.714079 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:11.714060    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/sub-test-model-a12988-kserve-5689fc6b77-kvffv"]
Jul 22 12:46:12.134560 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:12.134528    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b899067-2a30-4f2e-b442-472946e4ea0b" path="/var/lib/kubelet/pods/4b899067-2a30-4f2e-b442-472946e4ea0b/volumes"
Jul 22 12:46:12.712502 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:12.712473    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:12.712910 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:12.712527    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:12.725134 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:12.725109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:13.710459 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:13.710432    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:46:20.443846 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:20.443818    2565 scope.go:117] "RemoveContainer" containerID="78e9dc9899dbdf460b808143b3093639c3e2335255667212b130780eaa93f0e9"
Jul 22 12:46:32.629964 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:32.629927    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:32.951084 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:32.950992    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:33.022533 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:33.022501    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:36.404163 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:36.404131    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:46:45.987938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:45.987803    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:46.141954 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:46.141919    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:52.695190 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:52.695151    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:46:54.796153 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:54.796110    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:54.941102 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:54.941061    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:46:58.991922 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:46:58.991873    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:07.799162 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:07.799130    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:47:15.666156 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:15.666124    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:47:15.688081 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:15.688033    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:16.547233 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:16.547187    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:20.464522 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:20.464491    2565 scope.go:117] "RemoveContainer" containerID="0a1041b6c122d38c06f3484887c4255e0cbf3464ca64651d706630d9a18f69cf"
Jul 22 12:47:20.472618 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:20.472597    2565 scope.go:117] "RemoveContainer" containerID="ef0a9ce60ed1f839a9e045e77bc56a70564983094cc8e33380ea08daecfa7f5f"
Jul 22 12:47:22.695162 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:22.695126    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:47:30.686311 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:30.686251    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" podUID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" containerName="main" containerID="cri-o://91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf" gracePeriod=45
Jul 22 12:47:30.938550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:30.938480    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:47:31.008761 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008729    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvlnb\" (UniqueName: \"kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.008938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008771    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.008938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008812    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.008938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008866    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.008938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008893    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.008938 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.008913    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache\") pod \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\" (UID: \"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3\") "
Jul 22 12:47:31.009275 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.009115    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.009275 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.009150    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home" (OuterVolumeSpecName: "home") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.009275 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.009218    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.011026 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.010996    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:47:31.011026 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.010996    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb" (OuterVolumeSpecName: "kube-api-access-kvlnb") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "kube-api-access-kvlnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:31.011179 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.011041    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm" (OuterVolumeSpecName: "dshm") pod "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" (UID: "b1b7b3c7-2c40-457a-8524-ef4deb73b1d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:47:31.041359 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.041325    2565 generic.go:356] "Generic (PLEG): container finished" podID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" containerID="91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf" exitCode=0
Jul 22 12:47:31.041529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.041370    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"
Jul 22 12:47:31.041529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.041378    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" event={"ID":"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3","Type":"ContainerDied","Data":"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"}
Jul 22 12:47:31.041529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.041417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9" event={"ID":"b1b7b3c7-2c40-457a-8524-ef4deb73b1d3","Type":"ContainerDied","Data":"db7c272238c8d2fa654e514b46f81a55294da6f89fc4d943c6f07233d1786c7d"}
Jul 22 12:47:31.041529 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.041438    2565 scope.go:117] "RemoveContainer" containerID="91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"
Jul 22 12:47:31.050525 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.050508    2565 scope.go:117] "RemoveContainer" containerID="91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"
Jul 22 12:47:31.050817 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:47:31.050775    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf\": container with ID starting with 91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf not found: ID does not exist" containerID="91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"
Jul 22 12:47:31.050896 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.050826    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf"} err="failed to get container status \"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf\": rpc error: code = NotFound desc = could not find container \"91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf\": container with ID starting with 91837ae9dededcbccb06319c78b993263cb7594a1f769e6112e397d236b743cf not found: ID does not exist"
Jul 22 12:47:31.063898 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.063873    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:47:31.068799 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.068769    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a12988/rate-test-model-a12988-kserve-6696dfc6cf-wccf9"]
Jul 22 12:47:31.109468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109440    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tls-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.109468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109466    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-home\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.109613 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109475    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-model-cache\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.109613 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109484    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvlnb\" (UniqueName: \"kubernetes.io/projected/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-kube-api-access-kvlnb\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.109613 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109493    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-tmp-dir\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.109613 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:31.109502    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3-dshm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.134063 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:32.134026    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" path="/var/lib/kubelet/pods/b1b7b3c7-2c40-457a-8524-ef4deb73b1d3/volumes"
Jul 22 12:47:35.452916 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.452876    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"]
Jul 22 12:47:35.453401 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453357    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" containerName="main"
Jul 22 12:47:35.453401 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453376    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" containerName="main"
Jul 22 12:47:35.453523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453403    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b899067-2a30-4f2e-b442-472946e4ea0b" containerName="main"
Jul 22 12:47:35.453523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453412    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b899067-2a30-4f2e-b442-472946e4ea0b" containerName="main"
Jul 22 12:47:35.453523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453499    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b7b3c7-2c40-457a-8524-ef4deb73b1d3" containerName="main"
Jul 22 12:47:35.453523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.453509    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b899067-2a30-4f2e-b442-472946e4ea0b" containerName="main"
Jul 22 12:47:35.457963 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.457942    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:35.460245 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.460220    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:47:35.460368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.460226    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:47:35.466007 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.465981    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"]
Jul 22 12:47:35.538863 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.538823    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf\" (UID: \"47d22d67-1ac9-4a9a-8d98-6489a203c7f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:35.639384 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.639350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf\" (UID: \"47d22d67-1ac9-4a9a-8d98-6489a203c7f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:35.639993 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.639974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf\" (UID: \"47d22d67-1ac9-4a9a-8d98-6489a203c7f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:35.769577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.769546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:35.905403 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:35.905372    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"]
Jul 22 12:47:35.908915 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:47:35.908887    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d22d67_1ac9_4a9a_8d98_6489a203c7f5.slice/crio-00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419 WatchSource:0}: Error finding container 00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419: Status 404 returned error can't find the container with id 00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419
Jul 22 12:47:36.065499 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:36.065410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf" event={"ID":"47d22d67-1ac9-4a9a-8d98-6489a203c7f5","Type":"ContainerStarted","Data":"00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419"}
Jul 22 12:47:36.164830 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:36.164768    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:36.260589 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:36.260564    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:37.071142 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:37.071107    2565 generic.go:356] "Generic (PLEG): container finished" podID="47d22d67-1ac9-4a9a-8d98-6489a203c7f5" containerID="ee08f2d057aeaf54bd8e084a301ff10450e96ad03537b3a0b2b94674403e8e41" exitCode=0
Jul 22 12:47:37.071587 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:37.071187    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf" event={"ID":"47d22d67-1ac9-4a9a-8d98-6489a203c7f5","Type":"ContainerDied","Data":"ee08f2d057aeaf54bd8e084a301ff10450e96ad03537b3a0b2b94674403e8e41"}
Jul 22 12:47:38.211571 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:38.211545    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:38.258850 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:38.258822    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca\") pod \"47d22d67-1ac9-4a9a-8d98-6489a203c7f5\" (UID: \"47d22d67-1ac9-4a9a-8d98-6489a203c7f5\") "
Jul 22 12:47:38.259150 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:38.259128    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47d22d67-1ac9-4a9a-8d98-6489a203c7f5" (UID: "47d22d67-1ac9-4a9a-8d98-6489a203c7f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:47:38.360011 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:38.359915    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d22d67-1ac9-4a9a-8d98-6489a203c7f5-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:47:39.080196 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:39.080165    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"
Jul 22 12:47:39.080449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:39.080193    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf" event={"ID":"47d22d67-1ac9-4a9a-8d98-6489a203c7f5","Type":"ContainerDied","Data":"00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419"}
Jul 22 12:47:39.080449 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:39.080245    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00109fe73d98185e70d272fb6c0d3d8311686bf1774ff18dad42177fb3845419"
Jul 22 12:47:45.992913 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:45.992817    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:49.304413 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:49.304377    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:47:50.606335 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:50.606288    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:50.646629 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:50.646599    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"]
Jul 22 12:47:50.660920 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:50.660893    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a12988-wpmgf"]
Jul 22 12:47:50.752614 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:50.752569    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:51.094034 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:51.093998    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:52.133493 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:52.133456    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d22d67-1ac9-4a9a-8d98-6489a203c7f5" path="/var/lib/kubelet/pods/47d22d67-1ac9-4a9a-8d98-6489a203c7f5/volumes"
Jul 22 12:47:59.047239 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.047206    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:59.154921 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.154888    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:59.260884 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.260848    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:47:59.261108 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.261077    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" containerID="cri-o://b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4" gracePeriod=30
Jul 22 12:47:59.300065 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.299977    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:59.577112 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.577015    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:47:59.671448 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.671409    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"]
Jul 22 12:47:59.671844 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.671825    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d22d67-1ac9-4a9a-8d98-6489a203c7f5" containerName="revoke-keys"
Jul 22 12:47:59.671946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.671846    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d22d67-1ac9-4a9a-8d98-6489a203c7f5" containerName="revoke-keys"
Jul 22 12:47:59.671946 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.671924    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d22d67-1ac9-4a9a-8d98-6489a203c7f5" containerName="revoke-keys"
Jul 22 12:47:59.674727 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.674706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:47:59.677090 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.677071    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87ltg\""
Jul 22 12:47:59.677192 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.677071    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:47:59.684923 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.684900    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"]
Jul 22 12:47:59.720066 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.720037    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-980f27-pd46f\" (UID: \"002da5c6-018e-4295-9057-a38e0b73ac7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:47:59.820481 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.820452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-980f27-pd46f\" (UID: \"002da5c6-018e-4295-9057-a38e0b73ac7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:47:59.821046 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.821027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-980f27-pd46f\" (UID: \"002da5c6-018e-4295-9057-a38e0b73ac7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:47:59.985520 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:47:59.985436    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:48:00.326020 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:00.325926    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"]
Jul 22 12:48:00.328339 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:48:00.328309    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002da5c6_018e_4295_9057_a38e0b73ac7a.slice/crio-b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538 WatchSource:0}: Error finding container b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538: Status 404 returned error can't find the container with id b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538
Jul 22 12:48:00.443577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:00.443542    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:00.548398 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:00.548360    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:00.647262 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:00.647236    2565 patch_prober.go:28] interesting pod/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:48:00.647377 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:00.647285    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:48:01.178332 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:01.178303    2565 generic.go:356] "Generic (PLEG): container finished" podID="002da5c6-018e-4295-9057-a38e0b73ac7a" containerID="aa6572376f9f2f988f5c76ad5411c53bf85291f644044b8fd086f81d90890414" exitCode=0
Jul 22 12:48:01.178518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:01.178377    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f" event={"ID":"002da5c6-018e-4295-9057-a38e0b73ac7a","Type":"ContainerDied","Data":"aa6572376f9f2f988f5c76ad5411c53bf85291f644044b8fd086f81d90890414"}
Jul 22 12:48:01.178518 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:01.178414    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f" event={"ID":"002da5c6-018e-4295-9057-a38e0b73ac7a","Type":"ContainerStarted","Data":"b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538"}
Jul 22 12:48:02.316038 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:02.316014    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:48:02.342125 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:02.342100    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca\") pod \"002da5c6-018e-4295-9057-a38e0b73ac7a\" (UID: \"002da5c6-018e-4295-9057-a38e0b73ac7a\") "
Jul 22 12:48:02.342523 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:02.342497    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "002da5c6-018e-4295-9057-a38e0b73ac7a" (UID: "002da5c6-018e-4295-9057-a38e0b73ac7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:48:02.443352 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:02.443272    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002da5c6-018e-4295-9057-a38e0b73ac7a-maas-api-service-ca\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:03.192316 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:03.192282    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"
Jul 22 12:48:03.192513 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:03.192283    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f" event={"ID":"002da5c6-018e-4295-9057-a38e0b73ac7a","Type":"ContainerDied","Data":"b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538"}
Jul 22 12:48:03.192513 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:03.192418    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b09fa5037e8d8d9edf575cc4d4e1802e2115874bff7d5d99dbd8599fe74c9538"
Jul 22 12:48:04.506468 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.506444    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:48:04.560136 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560060    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjwtt\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560136 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560096    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560197    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560255    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560296    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560332    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560358    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560387    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560417    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560435    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560530    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:04.560577 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560541    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data\") pod \"e98c472c-5615-4e79-b00f-3f979d0b43ba\" (UID: \"e98c472c-5615-4e79-b00f-3f979d0b43ba\") "
Jul 22 12:48:04.560943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560841    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:48:04.560943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560908    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.560943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560923    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.560943 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.560935    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.561438 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.561409    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:04.562537 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.562508    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:48:04.562633 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.562539    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:04.562763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.562721    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:04.562763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.562749    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt" (OuterVolumeSpecName: "kube-api-access-tjwtt") pod "e98c472c-5615-4e79-b00f-3f979d0b43ba" (UID: "e98c472c-5615-4e79-b00f-3f979d0b43ba"). InnerVolumeSpecName "kube-api-access-tjwtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:04.662133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662102    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.662133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662131    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e98c472c-5615-4e79-b00f-3f979d0b43ba-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.662133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662141    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.662133 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662148    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.662395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662159    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjwtt\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-kube-api-access-tjwtt\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:04.662395 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:04.662171    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e98c472c-5615-4e79-b00f-3f979d0b43ba-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:05.202909 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.202868    2565 generic.go:356] "Generic (PLEG): container finished" podID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerID="b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4" exitCode=0
Jul 22 12:48:05.203091 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.202916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerDied","Data":"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"}
Jul 22 12:48:05.203091 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.202922    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"
Jul 22 12:48:05.203091 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.202977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74" event={"ID":"e98c472c-5615-4e79-b00f-3f979d0b43ba","Type":"ContainerDied","Data":"b3d6c7a5a428e357a66333f5deb2502ff8cb5fca1da9c16526632451b5643583"}
Jul 22 12:48:05.203091 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.203003    2565 scope.go:117] "RemoveContainer" containerID="b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"
Jul 22 12:48:05.217237 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.217206    2565 scope.go:117] "RemoveContainer" containerID="ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"
Jul 22 12:48:05.225200 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.225178    2565 scope.go:117] "RemoveContainer" containerID="b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"
Jul 22 12:48:05.225443 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:48:05.225424    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4\": container with ID starting with b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4 not found: ID does not exist" containerID="b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"
Jul 22 12:48:05.225492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.225452    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4"} err="failed to get container status \"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4\": rpc error: code = NotFound desc = could not find container \"b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4\": container with ID starting with b0c6cfb1b381accb22aec76c5ac3fe697dc3c8479288a31694fa5aae0c4d29e4 not found: ID does not exist"
Jul 22 12:48:05.225492 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.225469    2565 scope.go:117] "RemoveContainer" containerID="ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"
Jul 22 12:48:05.225707 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:48:05.225686    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9\": container with ID starting with ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9 not found: ID does not exist" containerID="ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"
Jul 22 12:48:05.225751 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.225714    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9"} err="failed to get container status \"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9\": rpc error: code = NotFound desc = could not find container \"ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9\": container with ID starting with ca2f37fd096b3314fef3574ddcf8aedd32324f2fb6eae04d822a2fecee6432c9 not found: ID does not exist"
Jul 22 12:48:05.229059 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.229036    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:48:05.238672 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:05.238649    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a12988-openshift-default-8595c99fbc-wvq74"]
Jul 22 12:48:06.134551 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:06.134513    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" path="/var/lib/kubelet/pods/e98c472c-5615-4e79-b00f-3f979d0b43ba/volumes"
Jul 22 12:48:08.733081 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:08.733050    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:10.260799 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:10.260761    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:13.104093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:13.104058    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2d5xw"]
Jul 22 12:48:14.757175 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:14.757137    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:14.768646 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:14.768616    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"]
Jul 22 12:48:14.772263 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:14.772237    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-980f27-pd46f"]
Jul 22 12:48:16.133216 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:16.133173    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="002da5c6-018e-4295-9057-a38e0b73ac7a" path="/var/lib/kubelet/pods/002da5c6-018e-4295-9057-a38e0b73ac7a/volumes"
Jul 22 12:48:16.376920 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:16.376887    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:22.003887 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.003843    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:22.070684 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.070654    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:22.264064 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.263986    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:48:22.264212 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.264189    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy" containerID="cri-o://ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9" gracePeriod=30
Jul 22 12:48:22.282235 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.282203    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:22.553306 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:22.553276    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:27.513812 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.513768    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:48:27.655068 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.654972    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655068 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655021    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655068 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655062    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655079    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655213    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655364 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8rxm\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm\") pod \"aaead397-d760-4888-ad8d-53ea330c4157\" (UID: \"aaead397-d760-4888-ad8d-53ea330c4157\") "
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655335    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655437    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655496    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-certs\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655510    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-credential-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.655683 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655524    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-workload-socket\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.655967 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.655730    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:48:27.656211 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.656184    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data" (OuterVolumeSpecName: "istio-data") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:27.657453 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.657432    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token" (OuterVolumeSpecName: "istio-token") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:27.657541 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.657527    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 12:48:27.657868 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.657849    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:48:27.657948 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.657925    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm" (OuterVolumeSpecName: "kube-api-access-n8rxm") pod "aaead397-d760-4888-ad8d-53ea330c4157" (UID: "aaead397-d760-4888-ad8d-53ea330c4157"). InnerVolumeSpecName "kube-api-access-n8rxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:27.756670 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756632    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaead397-d760-4888-ad8d-53ea330c4157-istiod-ca-cert\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.756670 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756663    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8rxm\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-kube-api-access-n8rxm\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.756670 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756678    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaead397-d760-4888-ad8d-53ea330c4157-istio-podinfo\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.757024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756697    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-data\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.757024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756709    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaead397-d760-4888-ad8d-53ea330c4157-istio-token\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:27.757024 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:27.756720    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaead397-d760-4888-ad8d-53ea330c4157-istio-envoy\") on node \"ip-10-0-132-143.ec2.internal\" DevicePath \"\""
Jul 22 12:48:28.308603 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.308573    2565 generic.go:356] "Generic (PLEG): container finished" podID="aaead397-d760-4888-ad8d-53ea330c4157" containerID="ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9" exitCode=0
Jul 22 12:48:28.308782 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.308622    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"
Jul 22 12:48:28.308782 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.308646    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerDied","Data":"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"}
Jul 22 12:48:28.308782 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.308701    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2" event={"ID":"aaead397-d760-4888-ad8d-53ea330c4157","Type":"ContainerDied","Data":"0845575dc6c70d27ffac6994a332c06cd769639b348bd9ec759b15f9b95071c7"}
Jul 22 12:48:28.308782 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.308726    2565 scope.go:117] "RemoveContainer" containerID="ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"
Jul 22 12:48:28.316964 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.316945    2565 scope.go:117] "RemoveContainer" containerID="3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"
Jul 22 12:48:28.324047 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.324031    2565 scope.go:117] "RemoveContainer" containerID="ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"
Jul 22 12:48:28.324295 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:48:28.324276    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9\": container with ID starting with ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9 not found: ID does not exist" containerID="ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"
Jul 22 12:48:28.324339 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.324305    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9"} err="failed to get container status \"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9\": rpc error: code = NotFound desc = could not find container \"ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9\": container with ID starting with ced69d91d709ed24cea8ad62f429e08cabf147a312857a69f1bd13f2d0bf0aa9 not found: ID does not exist"
Jul 22 12:48:28.324339 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.324322    2565 scope.go:117] "RemoveContainer" containerID="3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"
Jul 22 12:48:28.324554 ip-10-0-132-143 kubenswrapper[2565]: E0722 12:48:28.324535    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca\": container with ID starting with 3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca not found: ID does not exist" containerID="3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"
Jul 22 12:48:28.324617 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.324567    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca"} err="failed to get container status \"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca\": rpc error: code = NotFound desc = could not find container \"3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca\": container with ID starting with 3f624660c4d7a7944c9c0a1c515d6f13644856510e19cb8d52c1e7a20dbae0ca not found: ID does not exist"
Jul 22 12:48:28.328908 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.328886    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:48:28.336412 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.336392    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-980f27-openshift-default-7c567b6b67-55qr2"]
Jul 22 12:48:28.704702 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.704662    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x5744_d7825f8b-1508-4efa-a3a1-dde290b0a2e7/manager/0.log"
Jul 22 12:48:28.829860 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:28.829833    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lp4nc_0c985ceb-d49e-4073-98c2-07687d2cfaa0/manager/0.log"
Jul 22 12:48:29.081719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:29.081687    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z55d4_32a38fff-3a9c-4434-befc-9a776b417a17/server/0.log"
Jul 22 12:48:29.205432 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:29.205401    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nqnk8_ab892338-e44c-454c-b5e8-ba3b26f723e8/manager/0.log"
Jul 22 12:48:29.324924 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:29.324894    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-bgn22_d498609e-1203-469d-a9b0-4ea8f34ccb14/manager/0.log"
Jul 22 12:48:29.331370 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:29.331349    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-bgn22_d498609e-1203-469d-a9b0-4ea8f34ccb14/copy-manifests/0.log"
Jul 22 12:48:30.049592 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:30.049556    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745375-cxf8w_aec7b5d0-71ee-451d-b757-051825d57886/cleanup/0.log"
Jul 22 12:48:30.134241 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:30.134205    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaead397-d760-4888-ad8d-53ea330c4157" path="/var/lib/kubelet/pods/aaead397-d760-4888-ad8d-53ea330c4157/volumes"
Jul 22 12:48:30.163780 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:30.163751    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745390-kdt82_142410e0-9637-4434-8ae8-abb07a63c09b/cleanup/0.log"
Jul 22 12:48:32.380644 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:32.380610    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2d5xw_964dc4e7-fb39-4286-8831-48fe9214ca78/limitador/0.log"
Jul 22 12:48:32.504666 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:32.504623    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-59lpb_b42a81f1-0157-4aa5-bf2d-e54ac53cb868/manager/0.log"
Jul 22 12:48:32.861262 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:32.861233    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jv62k_07dc49e4-8a85-4d5a-a800-494cb4701627/istio-proxy/0.log"
Jul 22 12:48:32.988303 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:32.988271    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tjts8_962303b0-f3b0-40be-82af-979be562ac6c/discovery/0.log"
Jul 22 12:48:33.217567 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:33.217494    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f68f699cd-vtdjv_b51d5e0f-781c-46dd-b02a-8e137e7a7874/kube-auth-proxy/0.log"
Jul 22 12:48:39.852973 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:39.852929    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:39.951528 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:39.951490    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log" limit=1024
Jul 22 12:48:42.857589 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:42.857557    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mjqtp_f865ab54-07aa-4514-9ce8-64f633eec056/global-pull-secret-syncer/0.log"
Jul 22 12:48:43.001711 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:43.001677    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bkbtq_7f119165-7e05-4abd-b029-76446e4ae860/konnectivity-agent/0.log"
Jul 22 12:48:43.131209 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:43.131107    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-143.ec2.internal_bfaaa61dc548aab443d124ac4cc0e021/haproxy/0.log"
Jul 22 12:48:48.775244 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:48.775209    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2d5xw_964dc4e7-fb39-4286-8831-48fe9214ca78/limitador/0.log"
Jul 22 12:48:48.879817 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:48.879584    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-59lpb_b42a81f1-0157-4aa5-bf2d-e54ac53cb868/manager/0.log"
Jul 22 12:48:50.941575 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:50.941541    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2wh82_bba7f867-311c-41a7-a02a-22badc2d7c38/node-exporter/0.log"
Jul 22 12:48:50.992870 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:50.992842    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2wh82_bba7f867-311c-41a7-a02a-22badc2d7c38/kube-rbac-proxy/0.log"
Jul 22 12:48:51.013804 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:51.013767    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2wh82_bba7f867-311c-41a7-a02a-22badc2d7c38/init-textfile/0.log"
Jul 22 12:48:52.802046 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802010    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"]
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802296    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002da5c6-018e-4295-9057-a38e0b73ac7a" containerName="revoke-keys"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802307    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="002da5c6-018e-4295-9057-a38e0b73ac7a" containerName="revoke-keys"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802319    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802325    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802334    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802339    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802390    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802396    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802402    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="002da5c6-018e-4295-9057-a38e0b73ac7a" containerName="revoke-keys"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802409    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaead397-d760-4888-ad8d-53ea330c4157" containerName="istio-proxy"
Jul 22 12:48:52.802454 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.802413    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e98c472c-5615-4e79-b00f-3f979d0b43ba" containerName="istio-proxy"
Jul 22 12:48:52.805800 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.805767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.808165 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.808146    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s8cv\"/\"openshift-service-ca.crt\""
Jul 22 12:48:52.808260 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.808194    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s8cv\"/\"kube-root-ca.crt\""
Jul 22 12:48:52.808899 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.808884    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9s8cv\"/\"default-dockercfg-nssjf\""
Jul 22 12:48:52.817093 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.817075    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"]
Jul 22 12:48:52.848421 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.848392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-lib-modules\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.848550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.848426    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdsr2\" (UniqueName: \"kubernetes.io/projected/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-kube-api-access-rdsr2\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.848550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.848452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-sys\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.848550 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.848476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-proc\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.848695 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.848556    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-podres\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949220 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-lib-modules\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949220 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdsr2\" (UniqueName: \"kubernetes.io/projected/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-kube-api-access-rdsr2\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-sys\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-proc\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-sys\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-podres\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949411    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-lib-modules\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-podres\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.949464 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.949413    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-proc\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:52.959282 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:52.959257    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdsr2\" (UniqueName: \"kubernetes.io/projected/cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4-kube-api-access-rdsr2\") pod \"perf-node-gather-daemonset-vvsmj\" (UID: \"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4\") " pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:53.115304 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.115206    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:53.245426 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.245402    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"]
Jul 22 12:48:53.247824 ip-10-0-132-143 kubenswrapper[2565]: W0722 12:48:53.247781    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcdcb5d5f_6293_4198_9c2c_2e0c7c1c3cf4.slice/crio-bfa10bae459c399c33020033806c96b586981edc326625ca893067da2ea9fbb9 WatchSource:0}: Error finding container bfa10bae459c399c33020033806c96b586981edc326625ca893067da2ea9fbb9: Status 404 returned error can't find the container with id bfa10bae459c399c33020033806c96b586981edc326625ca893067da2ea9fbb9
Jul 22 12:48:53.423763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.423668    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj" event={"ID":"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4","Type":"ContainerStarted","Data":"3f4d1a6eefbcd809b72e4e0ed31b20e0367cb6d2572a81d014055e12488cf5ef"}
Jul 22 12:48:53.423763 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.423723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj" event={"ID":"cdcb5d5f-6293-4198-9c2c-2e0c7c1c3cf4","Type":"ContainerStarted","Data":"bfa10bae459c399c33020033806c96b586981edc326625ca893067da2ea9fbb9"}
Jul 22 12:48:53.424031 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.423819    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:53.443978 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:53.443929    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj" podStartSLOduration=1.4439141549999999 podStartE2EDuration="1.443914155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:53.441716636 +0000 UTC m=+2853.865147690" watchObservedRunningTime="2026-07-22 12:48:53.443914155 +0000 UTC m=+2853.867345209"
Jul 22 12:48:56.065042 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:56.065010    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmvx2_4f461d01-9d3e-4ba4-9dd7-8207c974e321/dns/0.log"
Jul 22 12:48:56.087597 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:56.087573    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmvx2_4f461d01-9d3e-4ba4-9dd7-8207c974e321/kube-rbac-proxy/0.log"
Jul 22 12:48:56.167368 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:56.167332    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmpph_8edb1ee9-df00-4ad9-b693-af21cdfac67e/dns-node-resolver/0.log"
Jul 22 12:48:56.762630 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:56.762579    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-58ss7_807e7e23-00a5-4e02-addf-9264b282db03/node-ca/0.log"
Jul 22 12:48:57.854089 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:57.854051    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jv62k_07dc49e4-8a85-4d5a-a800-494cb4701627/istio-proxy/0.log"
Jul 22 12:48:58.077444 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:58.077414    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tjts8_962303b0-f3b0-40be-82af-979be562ac6c/discovery/0.log"
Jul 22 12:48:58.123756 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:58.123677    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f68f699cd-vtdjv_b51d5e0f-781c-46dd-b02a-8e137e7a7874/kube-auth-proxy/0.log"
Jul 22 12:48:58.996920 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:58.996879    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7lb7c_81ed3d62-5302-4d22-aad4-b2aadc783b87/serve-healthcheck-canary/0.log"
Jul 22 12:48:59.437854 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:59.437811    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9s8cv/perf-node-gather-daemonset-vvsmj"
Jul 22 12:48:59.564662 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:59.564630    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-64xp7_af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc/kube-rbac-proxy/0.log"
Jul 22 12:48:59.588261 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:59.588237    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-64xp7_af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc/exporter/0.log"
Jul 22 12:48:59.613228 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:48:59.613206    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-64xp7_af99f0d9-bdde-4e7e-93f9-0db1c4e3bcdc/extractor/0.log"
Jul 22 12:49:02.450357 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:02.450203    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x5744_d7825f8b-1508-4efa-a3a1-dde290b0a2e7/manager/0.log"
Jul 22 12:49:02.796555 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:02.796517    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lp4nc_0c985ceb-d49e-4073-98c2-07687d2cfaa0/manager/0.log"
Jul 22 12:49:02.991313 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:02.991225    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z55d4_32a38fff-3a9c-4434-befc-9a776b417a17/server/0.log"
Jul 22 12:49:03.102103 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.102075    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nqnk8_ab892338-e44c-454c-b5e8-ba3b26f723e8/manager/0.log"
Jul 22 12:49:03.129858 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.129828    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-bgn22_d498609e-1203-469d-a9b0-4ea8f34ccb14/manager/0.log"
Jul 22 12:49:03.158331 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.158303    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-979495d87-bgn22_d498609e-1203-469d-a9b0-4ea8f34ccb14/copy-manifests/0.log"
Jul 22 12:49:03.604323 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.604292    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jv62k_07dc49e4-8a85-4d5a-a800-494cb4701627/istio-proxy/0.log"
Jul 22 12:49:03.817590 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.817556    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tjts8_962303b0-f3b0-40be-82af-979be562ac6c/discovery/0.log"
Jul 22 12:49:03.846719 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:03.846686    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f68f699cd-vtdjv_b51d5e0f-781c-46dd-b02a-8e137e7a7874/kube-auth-proxy/0.log"
Jul 22 12:49:05.116638 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:05.116608    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jv62k_07dc49e4-8a85-4d5a-a800-494cb4701627/istio-proxy/0.log"
Jul 22 12:49:05.343012 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:05.342984    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tjts8_962303b0-f3b0-40be-82af-979be562ac6c/discovery/0.log"
Jul 22 12:49:05.375876 ip-10-0-132-143 kubenswrapper[2565]: I0722 12:49:05.375778    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f68f699cd-vtdjv_b51d5e0f-781c-46dd-b02a-8e137e7a7874/kube-auth-proxy/0.log"