Jul 19 14:36:35.757671 ip-10-0-141-105 systemd[1]: Starting Kubernetes Kubelet...
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.159283    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163277    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163291    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163297    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163300    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163303    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163306    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163309    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.208732 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163312    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163314    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163332    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163335    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163338    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163341    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163344    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163347    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163350    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163352    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163355    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163358    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163360    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163363    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163365    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163368    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163370    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163376    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163381    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163384    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.215927 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163387    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163393    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163396    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163399    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163401    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163404    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163407    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163410    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163413    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163415    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163418    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163421    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163423    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163426    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163428    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163431    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163436    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163440    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163443    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.216573 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163446    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163450    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163452    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163455    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163457    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163460    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163463    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163466    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163468    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163470    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163474    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163476    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163479    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163482    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163484    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163487    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163490    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163493    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163495    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163498    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.217207 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163500    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163503    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163506    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163509    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163511    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163514    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163517    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163520    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163522    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163526    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163528    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163531    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163534    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163537    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163540    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163542    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163545    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163547    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163550    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163553    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.217865 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163556    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163558    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163562    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163565    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163575    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163578    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163580    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163583    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163586    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163590    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163593    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163597    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.163600    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164352    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164359    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164363    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164366    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164370    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164373    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164376    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.218513 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164379    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164382    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164385    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164387    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164390    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164393    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164402    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164405    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164409    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164413    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164417    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164420    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164423    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164426    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164428    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164431    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164433    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164436    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164439    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164441    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.219158 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164444    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164446    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164449    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164452    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164454    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164457    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164460    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164462    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164465    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164468    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164471    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164474    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164478    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164481    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164483    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164486    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164489    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164491    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164494    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164497    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.219819 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164500    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164503    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164506    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164509    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164511    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164514    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164517    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164520    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164522    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164526    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164530    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164533    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164536    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164539    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164542    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164545    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164548    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164550    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164553    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.220477 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164555    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164558    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164560    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164563    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164566    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164569    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164572    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164575    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164577    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164580    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164583    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164586    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164588    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164591    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164594    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164596    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164599    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164601    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164604    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.221150 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164606    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164608    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164611    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164613    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164616    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164618    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164621    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164623    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164626    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164628    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164634    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164637    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164640    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.164643    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165088    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165094    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165101    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165104    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165107    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165111    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.221838 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165113    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165116    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165120    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165124    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165127    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165130    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165134    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165137    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165140    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165142    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165145    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165147    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165150    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165152    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165155    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165157    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165160    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165162    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165165    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165167    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.222498 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165169    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165172    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165174    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165177    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165180    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165182    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165184    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165187    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165192    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165195    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165197    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165200    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165202    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165205    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165207    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165210    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165212    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165215    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165217    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.223152 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165220    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165223    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165225    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165227    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165230    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165232    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165235    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165237    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165240    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165242    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165245    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165248    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165251    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165253    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165256    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165258    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165261    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165265    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165268    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.223805 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165271    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165274    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165278    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165281    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165284    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165286    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165289    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165291    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165294    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165297    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165299    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165302    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165305    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165307    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165310    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165313    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165332    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165335    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165338    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165341    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.224670 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165343    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165346    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165348    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165351    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165354    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165356    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165359    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165362    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165364    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165367    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165369    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165372    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165375    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165377    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.165382    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165457    2585 flags.go:64] FLAG: --address="0.0.0.0"
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165465    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165471    2585 flags.go:64] FLAG: --anonymous-auth="true"
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165476    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165481    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 19 14:36:36.225335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165484    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165489    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165493    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165496    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165499    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165503    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165506    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165510    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165513    2585 flags.go:64] FLAG: --cgroup-root=""
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165517    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165520    2585 flags.go:64] FLAG: --client-ca-file=""
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165523    2585 flags.go:64] FLAG: --cloud-provider="external"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165526    2585 flags.go:64] FLAG: --cluster-dns="[]"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165531    2585 flags.go:64] FLAG: --cluster-domain=""
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165534    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165537    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165540    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165544    2585 flags.go:64] FLAG: --container-log-max-files="5"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165548    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165551    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165554    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165557    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165560    2585 flags.go:64] FLAG: --contention-profiling="false"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165563    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 19 14:36:36.225964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165566    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165569    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165574    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165579    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165582    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165585    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165588    2585 flags.go:64] FLAG: --enable-load-reader="false"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165591    2585 flags.go:64] FLAG: --enable-server="true"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165594    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165599    2585 flags.go:64] FLAG: --event-burst="100"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165603    2585 flags.go:64] FLAG: --event-qps="50"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165605    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165609    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165612    2585 flags.go:64] FLAG: --eviction-hard=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165615    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165619    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165623    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165627    2585 flags.go:64] FLAG: --eviction-soft=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165630    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165633    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165636    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165639    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165642    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165645    2585 flags.go:64] FLAG: --fail-swap-on="true"
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165648    2585 flags.go:64] FLAG: --feature-gates=""
Jul 19 14:36:36.226609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165652    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165655    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165658    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165662    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165665    2585 flags.go:64] FLAG: --healthz-port="10248"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165668    2585 flags.go:64] FLAG: --help="false"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165671    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-141-105.ec2.internal"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165674    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165677    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165682    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165686    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165689    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165692    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165696    2585 flags.go:64] FLAG: --image-service-endpoint=""
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165698    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165702    2585 flags.go:64] FLAG: --kube-api-burst="100"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165705    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165708    2585 flags.go:64] FLAG: --kube-api-qps="50"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165711    2585 flags.go:64] FLAG: --kube-reserved=""
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165714    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165717    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165720    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165723    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165725    2585 flags.go:64] FLAG: --lock-file=""
Jul 19 14:36:36.227691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165733    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165737    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165740    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165746    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165749    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165752    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165755    2585 flags.go:64] FLAG: --logging-format="text"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165757    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165761    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165764    2585 flags.go:64] FLAG: --manifest-url=""
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165767    2585 flags.go:64] FLAG: --manifest-url-header=""
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165771    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165774    2585 flags.go:64] FLAG: --max-open-files="1000000"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165779    2585 flags.go:64] FLAG: --max-pods="110"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165782    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165785    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165787    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165792    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165795    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165800    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165803    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165810    2585 flags.go:64] FLAG: --node-status-max-images="50"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165813    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165816    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 19 14:36:36.228345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165819    2585 flags.go:64] FLAG: --pod-cidr=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165822    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165828    2585 flags.go:64] FLAG: --pod-manifest-path=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165831    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165834    2585 flags.go:64] FLAG: --pods-per-core="0"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165837    2585 flags.go:64] FLAG: --port="10250"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165840    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165843    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05bb4190ea8d107d1"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165846    2585 flags.go:64] FLAG: --qos-reserved=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165850    2585 flags.go:64] FLAG: --read-only-port="10255"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165853    2585 flags.go:64] FLAG: --register-node="true"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165855    2585 flags.go:64] FLAG: --register-with-taints=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165859    2585 flags.go:64] FLAG: --registry-burst="10"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165862    2585 flags.go:64] FLAG: --registry-qps="5"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165865    2585 flags.go:64] FLAG: --reserved-cpus=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165868    2585 flags.go:64] FLAG: --reserved-memory=""
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165872    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165875    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165878    2585 flags.go:64] FLAG: --rotate-certificates="false"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165881    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165884    2585 flags.go:64] FLAG: --runonce="false"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165887    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165890    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165893    2585 flags.go:64] FLAG: --seccomp-default="false"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165895    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 19 14:36:36.229193 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165898    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165903    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165908    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165911    2585 flags.go:64] FLAG: --storage-driver-password="root"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165913    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165916    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165919    2585 flags.go:64] FLAG: --storage-driver-user="root"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165922    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165925    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165928    2585 flags.go:64] FLAG: --system-cgroups=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165930    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165936    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165939    2585 flags.go:64] FLAG: --tls-cert-file=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165942    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165947    2585 flags.go:64] FLAG: --tls-min-version=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165949    2585 flags.go:64] FLAG: --tls-private-key-file=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165952    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165956    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165959    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165962    2585 flags.go:64] FLAG: --v="2"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165966    2585 flags.go:64] FLAG: --version="false"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165970    2585 flags.go:64] FLAG: --vmodule=""
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165974    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.165977    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166097    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.293855 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166100    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166104    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166107    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166110    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166112    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166115    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166119    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166123    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166127    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166131    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166133    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166136    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166139    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166141    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166144    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166147    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166149    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166152    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166156    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.294911 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166160    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166163    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166166    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166168    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166171    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166174    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166177    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166180    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166182    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166185    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166187    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166190    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166192    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166195    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166197    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166200    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166202    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166205    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166208    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.295951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166210    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166213    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166217    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166221    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166223    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166226    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166229    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166232    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166234    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166237    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166239    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166242    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166245    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166247    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166249    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166252    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166254    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166257    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166260    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.296591 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166262    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166265    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166268    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166270    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166273    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166275    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166278    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166280    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166283    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166285    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166288    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166290    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166293    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166295    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166298    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166301    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166305    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166308    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166310    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166313    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.297635 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166315    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166333    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166336    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166339    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166341    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166344    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166347    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166349    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166352    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166355    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166357    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166360    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166363    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166365    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166369    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166372    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166375    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166379    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166381    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166384    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.298276 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.166387    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.166392    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.173422    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.173440    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.173459    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.173465    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173525    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173531    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173534    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173537    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173540    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173543    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173546    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173549    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.302072 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173552    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173556    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173560    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173563    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173566    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173569    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173571    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173574    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173578    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173582    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173586    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173588    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173592    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173594    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173597    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173600    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173603    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173605    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173608    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.302510 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173611    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173614    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173619    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173622    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173625    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173628    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173630    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173633    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173635    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173638    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173641    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173643    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173646    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173649    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173654    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173657    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173660    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173663    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173665    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173668    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.303788 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173670    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173673    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173676    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173678    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173681    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173684    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173687    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173690    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173692    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173696    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173698    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173701    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173704    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173708    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173713    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173716    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173718    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173721    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173724    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.304351 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173727    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173729    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173732    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173735    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173738    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173740    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173742    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173747    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173750    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173752    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173755    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173757    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173760    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173762    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173765    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173768    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173770    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173773    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173775    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173778    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.304924 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173782    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173784    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173787    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173789    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173792    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173795    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173797    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173801    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173804    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173806    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173809    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173811    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173814    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.173819    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:36:36.361346 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173949    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:36:36.316095 ip-10-0-141-105 systemd[1]: Started Kubernetes Kubelet.
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173954    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173957    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173960    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173963    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173967    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173970    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173973    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173975    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173978    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173980    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173983    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173986    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173988    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173991    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173993    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173996    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.173999    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174001    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174004    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:36:36.362163 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174007    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174009    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174012    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174014    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174017    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174019    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174023    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174026    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174028    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174031    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174033    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174036    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174038    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174042    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174046    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174048    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174051    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174055    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174058    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174060    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:36:36.362796 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174063    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174065    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174068    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174071    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174073    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174076    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174078    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174081    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174083    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174086    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174088    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174091    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174094    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174098    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174101    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174105    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174108    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174111    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174115    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:36:36.364149 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174118    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174121    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174124    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174126    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174129    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174131    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174134    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174136    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174139    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174141    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174146    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174148    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174151    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174153    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174156    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174158    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174161    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174163    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174166    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:36:36.364795 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174168    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174171    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174173    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174176    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174178    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174181    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174183    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174186    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174189    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174191    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174194    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174197    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174200    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174203    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174205    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174208    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174210    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174213    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174215    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174218    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:36:36.365476 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:36.174220    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.174225    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.174382    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.176437    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.177634    2585 server.go:1019] "Starting client certificate rotation"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.177736    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.178517    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.180249    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.184792    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.194932    2585 log.go:25] "Validated CRI v1 runtime API"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.197262    2585 log.go:25] "Validated CRI v1 image API"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.198194    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 19 14:36:36.367645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.200997    2585 fs.go:135] Filesystem UUIDs: map[19371b98-da08-42dc-a313-8b18759088bd:/dev/nvme0n1p3 25826092-a1f5-4d23-ae17-cff3c4b84b44:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 19 14:36:36.368062 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.201015    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.284381    2585 manager.go:216] Machine: {Timestamp:2026-07-19 14:36:36.28384819 +0000 UTC m=+0.424786655 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098607 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec283c78151fddc0e720e3f63e7ed93d SystemUUID:ec283c78-151f-ddc0-e720-e3f63e7ed93d BootID:7527bf73-ce52-4c4c-820d-c9efb0fafcaa Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:d0:50:90:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:d0:50:90:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:99:7f:02:12:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.284495    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.284652    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.284668    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.284747    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gkkzh"
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.285658    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 19 14:36:36.369124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.285682    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.286338    2585 topology_manager.go:138] "Creating topology manager with none policy"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.286349    2585 container_manager_linux.go:309] "Creating device plugin manager"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.286357    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.286371    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.287599    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.304999    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305148    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305183    2585 kubelet.go:486] "Attempting to sync node with API server"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305196    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305213    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305224    2585 kubelet.go:398] "Adding apiserver pod source"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305236    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305675    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.305694    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.308124    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.309962    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.309991    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311008    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311022    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311028    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311034    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311043    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311048    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311056    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311062    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311068    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311074    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311084    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311089    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311694    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.311702    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.314923    2585 server.go:1268] "Started kubelet"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.315772    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.317387    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.317426    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.317892    2585 server.go:310] "Adding debug handlers to kubelet server"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.318484    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.319832    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.321572    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:36.325911    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.326707    2585 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-105.ec2.internal" not found
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.326771    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.327135    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 19 14:36:36.369543 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:36.329544    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.330031    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.330039    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.330230    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.330290    2585 reconstruct.go:97] "Volume reconstruction finished"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.330296    2585 reconciler.go:29] "Reconciler: start to sync state"
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331072    2585 factory.go:55] Registering systemd factory
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331086    2585 factory.go:223] Registration of the systemd container factory successfully
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331263    2585 factory.go:153] Registering CRI-O factory
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331273    2585 factory.go:223] Registration of the crio container factory successfully
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331341    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331366    2585 factory.go:103] Registering Raw factory
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331381    2585 manager.go:1196] Started watching for new ooms in manager
Jul 19 14:36:36.371703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:36.331781    2585 manager.go:349] Starting recovery of all containers
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.191137    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-105.ec2.internal\" not found"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.191361    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.193737    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.201534    2585 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-105.ec2.internal" not found
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.201539    2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-105.ec2.internal\" not found" node="ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.203203    2585 manager.go:354] Recovery completed
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.208011    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.209917    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.209953    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.209969    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.210580    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.210588    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.210594    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.210605    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.211336    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.211363    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.211377    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.212878    2585 policy_none.go:49] "None policy: Start"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.212891    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.212902    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 19 14:36:38.215690 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.214890    2585 policy_none.go:47] "Start"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.259921    2585 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-105.ec2.internal" not found
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.268582    2585 manager.go:337] "Starting Device Plugin manager"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.268631    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.268647    2585 server.go:87] "Starting device plugin registration server"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.268984    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.268998    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.269072    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.269155    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.269164    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.269755    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 19 14:36:38.300736 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.269819    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-105.ec2.internal\" not found"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.380191    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.381309    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.381341    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.381351    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.381382    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.389053    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.394982 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.389079    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-105.ec2.internal\": node \"ip-10-0-141-105.ec2.internal\" not found"
Jul 19 14:36:38.470876 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.415512    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 19 14:36:38.470876 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.415581    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 19 14:36:38.470876 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.415607    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 19 14:36:38.470876 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:38.415647    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 19 14:36:38.470876 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.417824    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:38.538393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.516174    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"]
Jul 19 14:36:38.538393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.518900    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.538393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.520616    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.559465    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.572043    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.585396    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.592094    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.592179    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.592209    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.610663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.592237    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692560    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692599    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692620    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692666    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692693    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/666bf80ba1e1983d2d3dd851bc8ab130-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal\" (UID: \"666bf80ba1e1983d2d3dd851bc8ab130\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.696527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.692666    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d37fd728dd468b7ee1c820042e75b2d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-105.ec2.internal\" (UID: \"3d37fd728dd468b7ee1c820042e75b2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.875420 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.875387    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:38.891379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.891357    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 19 14:36:38.891510 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.891488    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:36:38.891576 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.891514    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:36:38.891576 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.891514    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:36:38.896047 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:38.896028    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal"
Jul 19 14:36:39.007958 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.007839    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-18 14:31:37 +0000 UTC" deadline="2027-12-21 07:24:45.608126532 +0000 UTC"
Jul 19 14:36:39.007958 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.007899    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12472h48m6.600230549s"
Jul 19 14:36:39.019904 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.019876    2585 apiserver.go:52] "Watching apiserver"
Jul 19 14:36:39.028184 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.026795    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 19 14:36:39.029033 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.029012    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-cjnrw","openshift-ovn-kubernetes/ovnkube-node-lrwlt","kube-system/global-pull-secret-syncer-jjgj4","openshift-cluster-node-tuning-operator/tuned-556b6","openshift-dns/node-resolver-48n64","openshift-image-registry/node-ca-hm8sq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal","openshift-multus/network-metrics-daemon-vftg8","kube-system/konnectivity-agent-5t7hb","kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5","openshift-multus/multus-additional-cni-plugins-h4cnb","openshift-multus/multus-q49nn","openshift-network-diagnostics/network-check-target-57tbc"]
Jul 19 14:36:39.031462 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.031442    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.033665 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.033645    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bx6tl\""
Jul 19 14:36:39.033772 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.033650    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.033895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.033881    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.034048 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.034022    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.034190 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.034169    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 19 14:36:39.036114 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036095    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 19 14:36:39.036222 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036116    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.036222 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036145    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 19 14:36:39.036222 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.036192    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:39.036913 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036895    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 19 14:36:39.036913 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036905    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6c9kd\""
Jul 19 14:36:39.037004 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036897    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.037004 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036936    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 19 14:36:39.037004 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.036966    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.038215 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.038201    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.040088 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.040069    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.040176 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.040075    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8z9bb\""
Jul 19 14:36:39.040176 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.040134    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.040327 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.040309    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.042160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.042145    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-527sj\""
Jul 19 14:36:39.042376 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.042362    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.042465 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.042435    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.042465 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.042443    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.044030 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044013    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 19 14:36:39.044378 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044361    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rgq7n\""
Jul 19 14:36:39.044448 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044361    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.044448 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044442    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.044599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044382    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 19 14:36:39.044797 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.044766    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.044878 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.044856    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:39.046830 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.046817    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.048971 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.048957    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rwj4z\""
Jul 19 14:36:39.049047 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.049032    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.049134 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.049111    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 19 14:36:39.049231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.049164    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 19 14:36:39.053602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.051530    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.053602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.051582    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 19 14:36:39.053602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.051583    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.053602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.051676    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.053602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.051533    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ktwws\""
Jul 19 14:36:39.053856 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.053801    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 19 14:36:39.054174 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054153    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 19 14:36:39.054371 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054351    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggpxf\""
Jul 19 14:36:39.054480 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054463    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 19 14:36:39.054526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054480    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 19 14:36:39.054558 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054539    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 19 14:36:39.054595 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054542    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:36:39.054725 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.054685    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.056706 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.056689    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 19 14:36:39.056792 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.056763    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zs8z7\""
Jul 19 14:36:39.057044 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.057027    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:39.057118 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.057088    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:39.081570 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.081550    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nrztm"
Jul 19 14:36:39.087090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.087070    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nrztm"
Jul 19 14:36:39.093909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.093891    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-kubelet-config\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.093970 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.093916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4957289c-6405-4f3f-8531-5099ea2163ae-host\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.093970 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.093935    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-modprobe-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.093970 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.093957    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysconfig\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094099 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.093977    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-kubernetes\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094099 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094020    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-host\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094099 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094066    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6b1cf3-bddb-4991-a759-11fb984504be-tmp-dir\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094096    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-log-socket\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094119    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8add3f-5454-4a1e-be52-f3131d364a2d-konnectivity-ca\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094135    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094151    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-kubelet\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094166    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-var-lib-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094188    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-node-log\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-config\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094251    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-systemd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094272    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094294    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prvbr\" (UniqueName: \"kubernetes.io/projected/6c96766b-2429-4fa1-ae33-92005de587c2-kube-api-access-prvbr\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094309    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-systemd\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094325    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-run\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094341    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-tmp\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094373    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-netns\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094404    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-env-overrides\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094466    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-conf\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094489    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4nss\" (UniqueName: \"kubernetes.io/projected/cb6b1cf3-bddb-4991-a759-11fb984504be-kube-api-access-v4nss\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decdacd9-6e79-40b5-8cb5-32217e009188-host-slash\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094536    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4957289c-6405-4f3f-8531-5099ea2163ae-serviceca\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094575    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndrm2\" (UniqueName: \"kubernetes.io/projected/223f7083-66c8-4efb-9c2e-f19a4b4724be-kube-api-access-ndrm2\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094603    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-slash\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094624    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-netd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094646    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c96766b-2429-4fa1-ae33-92005de587c2-ovn-node-metrics-cert\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094668    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094690    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb6b1cf3-bddb-4991-a759-11fb984504be-hosts-file\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094718    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szvnb\" (UniqueName: \"kubernetes.io/projected/decdacd9-6e79-40b5-8cb5-32217e009188-kube-api-access-szvnb\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094756    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-lib-modules\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.094824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094798    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-etc-tuned\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8add3f-5454-4a1e-be52-f3131d364a2d-agent-certs\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094832    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-dbus\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094849    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-systemd-units\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094871    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-ovn\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094887    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-bin\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094913    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-script-lib\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094941    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-var-lib-kubelet\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094968    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/decdacd9-6e79-40b5-8cb5-32217e009188-iptables-alerter-script\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.094991    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd4jl\" (UniqueName: \"kubernetes.io/projected/4957289c-6405-4f3f-8531-5099ea2163ae-kube-api-access-bd4jl\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.095007    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.095021    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-etc-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.095041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-sys\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.095206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.095056    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h29j4\" (UniqueName: \"kubernetes.io/projected/3807bf89-4f35-43e9-b170-b252aed8545a-kube-api-access-h29j4\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.145685 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.145663    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 19 14:36:39.195581 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195538    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.195581 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195579    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cnibin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szvnb\" (UniqueName: \"kubernetes.io/projected/decdacd9-6e79-40b5-8cb5-32217e009188-kube-api-access-szvnb\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-lib-modules\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195717    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8add3f-5454-4a1e-be52-f3131d364a2d-agent-certs\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195739    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-dbus\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-ovn\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195777    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-lib-modules\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195799    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-bin\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.195835 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195822    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-script-lib\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195848    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-ovn\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195850    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195876    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-system-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195890    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-bin\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195952    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-socket-dir-parent\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195995    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd4jl\" (UniqueName: \"kubernetes.io/projected/4957289c-6405-4f3f-8531-5099ea2163ae-kube-api-access-bd4jl\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.195966    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-dbus\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196023    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-sys\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196049    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-registration-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196089    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-kubelet-config\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196106    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-sys\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196105    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196112    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4957289c-6405-4f3f-8531-5099ea2163ae-host\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196145    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4957289c-6405-4f3f-8531-5099ea2163ae-host\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.196205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196158    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d62cd493-14c5-4c1c-aef3-ce86ea253b26-kubelet-config\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196177    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-modprobe-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196197    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-host\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prvbr\" (UniqueName: \"kubernetes.io/projected/6c96766b-2429-4fa1-ae33-92005de587c2-kube-api-access-prvbr\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-log-socket\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196256    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-var-lib-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cnibin\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196297    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-modprobe-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196308    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-os-release\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196314    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-log-socket\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196280    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-host\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196323    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-var-lib-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196345    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-multus-certs\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-run\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.196814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196400    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-device-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196425    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-netns\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196442    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-script-lib\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196449    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-bin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196440    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-run\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196480    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-conf\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196497    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-conf-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decdacd9-6e79-40b5-8cb5-32217e009188-host-slash\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196564    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4957289c-6405-4f3f-8531-5099ea2163ae-serviceca\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196578    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/decdacd9-6e79-40b5-8cb5-32217e009188-host-slash\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196589    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndrm2\" (UniqueName: \"kubernetes.io/projected/223f7083-66c8-4efb-9c2e-f19a4b4724be-kube-api-access-ndrm2\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196614    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-conf\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196630    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb6b1cf3-bddb-4991-a759-11fb984504be-hosts-file\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196654    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-hostroot\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.197703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196679    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-etc-tuned\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196701    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-systemd-units\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196706    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb6b1cf3-bddb-4991-a759-11fb984504be-hosts-file\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-var-lib-kubelet\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196754    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196778    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-multus\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-systemd-units\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196829    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-var-lib-kubelet\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/decdacd9-6e79-40b5-8cb5-32217e009188-iptables-alerter-script\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196879    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196901    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-etc-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.196975    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196998    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-etc-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.196951    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h29j4\" (UniqueName: \"kubernetes.io/projected/3807bf89-4f35-43e9-b170-b252aed8545a-kube-api-access-h29j4\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197019    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4957289c-6405-4f3f-8531-5099ea2163ae-serviceca\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.197061    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:36:39.697032149 +0000 UTC m=+2.123509146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197102    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-k8s-cni-cncf-io\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197128    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-kubelet\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197153    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-daemon-config\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197176    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-etc-kubernetes\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4nss\" (UniqueName: \"kubernetes.io/projected/cb6b1cf3-bddb-4991-a759-11fb984504be-kube-api-access-v4nss\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197233    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysconfig\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197287    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysconfig\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197343    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/decdacd9-6e79-40b5-8cb5-32217e009188-iptables-alerter-script\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197348    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-kubernetes\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197386    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-kubernetes\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197409    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6b1cf3-bddb-4991-a759-11fb984504be-tmp-dir\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.198908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197438    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6qlq\" (UniqueName: \"kubernetes.io/projected/b92bec5f-8091-47b8-bfcd-f0d15fec6559-kube-api-access-l6qlq\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197465    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197490    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8add3f-5454-4a1e-be52-f3131d364a2d-konnectivity-ca\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197515    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197539    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-kubelet\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197560    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-node-log\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197632    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-node-log\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197631    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-config\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.197654    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197672    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7bj\" (UniqueName: \"kubernetes.io/projected/c066d4f2-b0cf-4a09-8306-901378c8ac79-kube-api-access-vx7bj\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197686    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb6b1cf3-bddb-4991-a759-11fb984504be-tmp-dir\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197697    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-system-cni-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.197715    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:39.697699723 +0000 UTC m=+2.124176731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:39.199561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197757    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-openvswitch\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197768    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-systemd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197812    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-kubelet\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197816    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197846    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-systemd\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197851    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-run-systemd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197871    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-tmp\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197892    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197909    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-systemd\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-socket-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.197970    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-sys-fs\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198117    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-ovnkube-config\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8add3f-5454-4a1e-be52-f3131d364a2d-konnectivity-ca\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198164    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.200231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198214    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-netns\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198242    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198266    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-env-overrides\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198289    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-netns\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198294    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198290    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198360    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-os-release\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198401    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198426    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cni-binary-copy\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198450    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4fhg\" (UniqueName: \"kubernetes.io/projected/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-kube-api-access-t4fhg\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198476    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-slash\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198502    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-netd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198535    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c96766b-2429-4fa1-ae33-92005de587c2-ovn-node-metrics-cert\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198559    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.200936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198566    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-slash\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198588    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c96766b-2429-4fa1-ae33-92005de587c2-host-cni-netd\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198665    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c96766b-2429-4fa1-ae33-92005de587c2-env-overrides\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.198703    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3807bf89-4f35-43e9-b170-b252aed8545a-etc-sysctl-d\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.199633    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-etc-tuned\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.199917    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8add3f-5454-4a1e-be52-f3131d364a2d-agent-certs\") pod \"konnectivity-agent-5t7hb\" (UID: \"ed8add3f-5454-4a1e-be52-f3131d364a2d\") " pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.200029    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3807bf89-4f35-43e9-b170-b252aed8545a-tmp\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.201711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.201544    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c96766b-2429-4fa1-ae33-92005de587c2-ovn-node-metrics-cert\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.203417 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.203385    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd4jl\" (UniqueName: \"kubernetes.io/projected/4957289c-6405-4f3f-8531-5099ea2163ae-kube-api-access-bd4jl\") pod \"node-ca-hm8sq\" (UID: \"4957289c-6405-4f3f-8531-5099ea2163ae\") " pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.203604 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.203587    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prvbr\" (UniqueName: \"kubernetes.io/projected/6c96766b-2429-4fa1-ae33-92005de587c2-kube-api-access-prvbr\") pod \"ovnkube-node-lrwlt\" (UID: \"6c96766b-2429-4fa1-ae33-92005de587c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.203809 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.203772    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szvnb\" (UniqueName: \"kubernetes.io/projected/decdacd9-6e79-40b5-8cb5-32217e009188-kube-api-access-szvnb\") pod \"iptables-alerter-cjnrw\" (UID: \"decdacd9-6e79-40b5-8cb5-32217e009188\") " pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.204541 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.204526    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndrm2\" (UniqueName: \"kubernetes.io/projected/223f7083-66c8-4efb-9c2e-f19a4b4724be-kube-api-access-ndrm2\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.207641 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.207623    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4nss\" (UniqueName: \"kubernetes.io/projected/cb6b1cf3-bddb-4991-a759-11fb984504be-kube-api-access-v4nss\") pod \"node-resolver-48n64\" (UID: \"cb6b1cf3-bddb-4991-a759-11fb984504be\") " pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.207725 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.207692    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h29j4\" (UniqueName: \"kubernetes.io/projected/3807bf89-4f35-43e9-b170-b252aed8545a-kube-api-access-h29j4\") pod \"tuned-556b6\" (UID: \"3807bf89-4f35-43e9-b170-b252aed8545a\") " pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.299066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.298968    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-sys-fs\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.299066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.299066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299028    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.299066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299055    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-os-release\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.299066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299070    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299085    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cni-binary-copy\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299102    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4fhg\" (UniqueName: \"kubernetes.io/projected/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-kube-api-access-t4fhg\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299111    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-sys-fs\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299121    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299232    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299269    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299304    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cnibin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299347    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299398    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-system-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299423    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-socket-dir-parent\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299441    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-os-release\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.299442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299451    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-registration-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299480    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-system-cni-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299499    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-registration-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cnibin\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-os-release\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299538    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-socket-dir-parent\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299539    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cnibin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-multus-certs\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299587    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-device-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299590    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-multus-certs\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299600    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-os-release\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299613    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-netns\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299636    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cnibin\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299639    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-bin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299614    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299651    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-device-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299651    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-netns\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299696    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-bin\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-conf-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-hostroot\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299774    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-conf-dir\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299777    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299803    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-hostroot\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299746    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299803    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299821    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-cni-binary-copy\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299849    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-multus\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299883    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-k8s-cni-cncf-io\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299905    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b92bec5f-8091-47b8-bfcd-f0d15fec6559-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.300638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299913    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-cni-multus\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-kubelet\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299926    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299939    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-run-k8s-cni-cncf-io\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299960    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-host-var-lib-kubelet\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-daemon-config\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.299987    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-etc-kubernetes\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300010    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6qlq\" (UniqueName: \"kubernetes.io/projected/b92bec5f-8091-47b8-bfcd-f0d15fec6559-kube-api-access-l6qlq\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300033    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300047    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-etc-kubernetes\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300065    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7bj\" (UniqueName: \"kubernetes.io/projected/c066d4f2-b0cf-4a09-8306-901378c8ac79-kube-api-access-vx7bj\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300082    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-system-cni-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300106    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-socket-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300164    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b92bec5f-8091-47b8-bfcd-f0d15fec6559-system-cni-dir\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.301117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300262    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c066d4f2-b0cf-4a09-8306-901378c8ac79-socket-dir\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.301498 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.300363    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-multus-daemon-config\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.305581 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.305557    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:39.305581 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.305582    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:39.305724 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.305596    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:39.305724 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.305664    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:39.805646051 +0000 UTC m=+2.232123060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:39.307592 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.307568    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4fhg\" (UniqueName: \"kubernetes.io/projected/bd2f75cd-6470-4013-b544-d9f2b9ffdc37-kube-api-access-t4fhg\") pod \"multus-q49nn\" (UID: \"bd2f75cd-6470-4013-b544-d9f2b9ffdc37\") " pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.307592 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.307590    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7bj\" (UniqueName: \"kubernetes.io/projected/c066d4f2-b0cf-4a09-8306-901378c8ac79-kube-api-access-vx7bj\") pod \"aws-ebs-csi-driver-node-5dtd5\" (UID: \"c066d4f2-b0cf-4a09-8306-901378c8ac79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.307731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.307623    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6qlq\" (UniqueName: \"kubernetes.io/projected/b92bec5f-8091-47b8-bfcd-f0d15fec6559-kube-api-access-l6qlq\") pod \"multus-additional-cni-plugins-h4cnb\" (UID: \"b92bec5f-8091-47b8-bfcd-f0d15fec6559\") " pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.358573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.358541    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjnrw"
Jul 19 14:36:39.376902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.376872    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:36:39.395602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.395577    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-556b6"
Jul 19 14:36:39.420245 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.420217    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-48n64"
Jul 19 14:36:39.425646 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.425594    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hm8sq"
Jul 19 14:36:39.440045 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.440010    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:36:39.443809 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.443769    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5"
Jul 19 14:36:39.454922 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.454760    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:36:39.456395 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.456371    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc066d4f2_b0cf_4a09_8306_901378c8ac79.slice/crio-f5b046da47b013dbd8e92d57a428f9d93cd8c068971a129289c01a4b8cfdf5d2 WatchSource:0}: Error finding container f5b046da47b013dbd8e92d57a428f9d93cd8c068971a129289c01a4b8cfdf5d2: Status 404 returned error can't find the container with id f5b046da47b013dbd8e92d57a428f9d93cd8c068971a129289c01a4b8cfdf5d2
Jul 19 14:36:39.458565 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.458531    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h4cnb"
Jul 19 14:36:39.463433 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.463264    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q49nn"
Jul 19 14:36:39.469371 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.469344    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92bec5f_8091_47b8_bfcd_f0d15fec6559.slice/crio-c9cc3a5874ebe72f0b31af990db8bcd2b1aff38ea2eb792806879f0cd7cc2177 WatchSource:0}: Error finding container c9cc3a5874ebe72f0b31af990db8bcd2b1aff38ea2eb792806879f0cd7cc2177: Status 404 returned error can't find the container with id c9cc3a5874ebe72f0b31af990db8bcd2b1aff38ea2eb792806879f0cd7cc2177
Jul 19 14:36:39.472055 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.472029    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2f75cd_6470_4013_b544_d9f2b9ffdc37.slice/crio-b2014647266c69d1fcd26226d5d8bcdeab256dedb63959b42d1a1db5c59d40f7 WatchSource:0}: Error finding container b2014647266c69d1fcd26226d5d8bcdeab256dedb63959b42d1a1db5c59d40f7: Status 404 returned error can't find the container with id b2014647266c69d1fcd26226d5d8bcdeab256dedb63959b42d1a1db5c59d40f7
Jul 19 14:36:39.630537 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.630495    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecdacd9_6e79_40b5_8cb5_32217e009188.slice/crio-2bcd42039a124123dfd292010fd877ef7dacae1d90a623616f61d56ae7ee98cd WatchSource:0}: Error finding container 2bcd42039a124123dfd292010fd877ef7dacae1d90a623616f61d56ae7ee98cd: Status 404 returned error can't find the container with id 2bcd42039a124123dfd292010fd877ef7dacae1d90a623616f61d56ae7ee98cd
Jul 19 14:36:39.659671 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.659630    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c96766b_2429_4fa1_ae33_92005de587c2.slice/crio-975566048eb2125f79a60a059e4e3b3f0ef9e7becab0d39ca95abd03dae4e702 WatchSource:0}: Error finding container 975566048eb2125f79a60a059e4e3b3f0ef9e7becab0d39ca95abd03dae4e702: Status 404 returned error can't find the container with id 975566048eb2125f79a60a059e4e3b3f0ef9e7becab0d39ca95abd03dae4e702
Jul 19 14:36:39.702524 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.702493    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:39.702694 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.702547    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:39.702694 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.702662    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:39.702694 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.702669    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:39.702836 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.702719    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:40.702705845 +0000 UTC m=+3.129182837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:39.702836 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.702733    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:36:40.702726633 +0000 UTC m=+3.129203623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:39.727989 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.727963    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:39.774610 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:36:39.774567    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3807bf89_4f35_43e9_b170_b252aed8545a.slice/crio-c41e62a2bf0d80f55fd91f622a3016bb73deaaba38f2f837c991485ded4d68a8 WatchSource:0}: Error finding container c41e62a2bf0d80f55fd91f622a3016bb73deaaba38f2f837c991485ded4d68a8: Status 404 returned error can't find the container with id c41e62a2bf0d80f55fd91f622a3016bb73deaaba38f2f837c991485ded4d68a8
Jul 19 14:36:39.894156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.894078    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:39.904416 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:39.904301    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:39.904594 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.904544    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:39.904594 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.904566    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:39.904701 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.904596    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:39.904701 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:39.904661    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:40.904641113 +0000 UTC m=+3.331118125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:40.090419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.090342    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:31:39 +0000 UTC" deadline="2028-04-05 11:33:27.454886118 +0000 UTC"
Jul 19 14:36:40.090419 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.090418    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15020h56m47.364473055s"
Jul 19 14:36:40.120326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.120289    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:36:40.418023 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.417990    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:40.418216 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.418136    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:40.443366 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.443303    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-556b6" event={"ID":"3807bf89-4f35-43e9-b170-b252aed8545a","Type":"ContainerStarted","Data":"c41e62a2bf0d80f55fd91f622a3016bb73deaaba38f2f837c991485ded4d68a8"}
Jul 19 14:36:40.475207 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.474828    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"975566048eb2125f79a60a059e4e3b3f0ef9e7becab0d39ca95abd03dae4e702"}
Jul 19 14:36:40.488343 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.488307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjnrw" event={"ID":"decdacd9-6e79-40b5-8cb5-32217e009188","Type":"ContainerStarted","Data":"2bcd42039a124123dfd292010fd877ef7dacae1d90a623616f61d56ae7ee98cd"}
Jul 19 14:36:40.499114 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.499024    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerStarted","Data":"c9cc3a5874ebe72f0b31af990db8bcd2b1aff38ea2eb792806879f0cd7cc2177"}
Jul 19 14:36:40.514221 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.514182    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5" event={"ID":"c066d4f2-b0cf-4a09-8306-901378c8ac79","Type":"ContainerStarted","Data":"f5b046da47b013dbd8e92d57a428f9d93cd8c068971a129289c01a4b8cfdf5d2"}
Jul 19 14:36:40.528500 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.527938    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-48n64" event={"ID":"cb6b1cf3-bddb-4991-a759-11fb984504be","Type":"ContainerStarted","Data":"651827d590c6617759ebf3b664358ab46af0ed489866a7c78203477180971738"}
Jul 19 14:36:40.546678 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.546536    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q49nn" event={"ID":"bd2f75cd-6470-4013-b544-d9f2b9ffdc37","Type":"ContainerStarted","Data":"b2014647266c69d1fcd26226d5d8bcdeab256dedb63959b42d1a1db5c59d40f7"}
Jul 19 14:36:40.559186 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.559147    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5t7hb" event={"ID":"ed8add3f-5454-4a1e-be52-f3131d364a2d","Type":"ContainerStarted","Data":"f18eef8e037002b4fc455eb1b89a9827abc6f4ae7754f4a3e9a49e87427247e5"}
Jul 19 14:36:40.573943 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.573881    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hm8sq" event={"ID":"4957289c-6405-4f3f-8531-5099ea2163ae","Type":"ContainerStarted","Data":"2fbd061be33425800ea68c60cb00b1f2383ccac5e49d6ca73881403ce8cc354a"}
Jul 19 14:36:40.606774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.606737    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerStarted","Data":"57f67187cc31a0aaca830fb967c1abd9b137b8d563905676113f591ec5b999a6"}
Jul 19 14:36:40.625384 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.625082    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" event={"ID":"3d37fd728dd468b7ee1c820042e75b2d","Type":"ContainerStarted","Data":"55a0bebc16f19d43ca085e26c69f566db1f46edea48bb00be4addd666ebfa53a"}
Jul 19 14:36:40.721141 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.721094    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:40.721338 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.721165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:40.721338 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.721307    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:40.721451 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.721372    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:42.721353058 +0000 UTC m=+5.147830065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:40.721725 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.721604    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:40.721725 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.721660    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:36:42.721644671 +0000 UTC m=+5.148121675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:40.923583 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:40.923492    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:40.923838 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.923674    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:40.923838 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.923696    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:40.923838 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.923709    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:40.923838 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:40.923769    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:42.923750869 +0000 UTC m=+5.350227883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:41.091236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:41.091167    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:31:39 +0000 UTC" deadline="2028-03-23 10:46:03.15216458 +0000 UTC"
Jul 19 14:36:41.091236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:41.091231    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14708h9m22.060937105s"
Jul 19 14:36:41.416457 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:41.416376    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:41.416637 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:41.416516    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:41.417000 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:41.416979    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:41.417122 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:41.417100    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:42.419896 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:42.419374    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:42.419896 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.419511    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:42.739962 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:42.739836    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:42.739962 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:42.739907    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:42.740210 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.740067    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:42.740210 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.740152    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:46.740132477 +0000 UTC m=+9.166609467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:42.740642 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.740615    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:42.740776 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.740678    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:36:46.740660442 +0000 UTC m=+9.167137433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:42.941126 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:42.941086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:42.941377 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.941259    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:42.941377 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.941274    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:42.941377 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.941284    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:42.941377 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:42.941334    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:46.941318384 +0000 UTC m=+9.367795376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:43.416830 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:43.416763    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:43.417032 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:43.416954    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:43.417032 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:43.416994    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:43.417142 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:43.417109    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:44.420749 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:44.420257    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:44.420749 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:44.420386    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:45.416476 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:45.416436    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:45.416679 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:45.416577    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:45.416679 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:45.416656    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:45.416815 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:45.416777    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:46.420159 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:46.420068    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:46.420574 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.420174    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:46.775840 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:46.775731    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:46.775840 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:46.775806    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:46.776043 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.775917    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:46.776043 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.775957    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:46.776043 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.775996    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:36:54.775974167 +0000 UTC m=+17.202451160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:46.776043 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.776015    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:54.77600608 +0000 UTC m=+17.202483072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:46.977820 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:46.977656    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:46.978014 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.977842    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:46.978014 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.977864    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:46.978014 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.977877    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:46.978014 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:46.977943    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:36:54.977923136 +0000 UTC m=+17.404400128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:47.416198 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:47.416155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:47.416388 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:47.416304    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:47.416671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:47.416648    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:47.416821 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:47.416770    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:48.417635 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:48.417587    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:48.418126 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:48.417716    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:49.416026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:49.415982    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:49.416026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:49.416020    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:49.416287 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:49.416139    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:49.416346 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:49.416325    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:50.416855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:50.416812    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:50.417350 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:50.416948    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:51.416905 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:51.416800    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:51.416905 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:51.416829    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:51.417421 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:51.416939    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:51.417421 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:51.417103    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:52.416057 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:52.416028    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:52.416220 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:52.416143    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:53.416600 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:53.416564    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:53.417050 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:53.416678    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:53.417050 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:53.416742    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:53.417050 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:53.416879    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:54.416898 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:54.416858    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:54.417350 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:54.417000    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:54.839445 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:54.839362    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:54.839445 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:54.839422    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:54.839636 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:54.839537    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:54.839636 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:54.839597    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:37:10.839584291 +0000 UTC m=+33.266061282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:36:54.839636 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:54.839537    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:54.839764 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:54.839671    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:37:10.839657229 +0000 UTC m=+33.266134227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:36:55.040874 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:55.040841    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:55.041031 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.041008    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:36:55.041069 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.041032    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:36:55.041069 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.041042    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:55.041135 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.041095    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:37:11.041081607 +0000 UTC m=+33.467558598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:36:55.416098 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:55.416063    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:55.416098 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:55.416082    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:55.416373 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.416206    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:55.416431 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:55.416361    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:56.416486 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:56.416266    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:56.416946 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:56.416589    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:57.416123 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:57.416086    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:57.416329 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:57.416096    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:57.416329 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:57.416230    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:57.416329 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:57.416286    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:36:58.416992 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:58.416955    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:36:58.417471 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:58.417077    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:36:59.415802 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:59.415765    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:36:59.415894 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:36:59.415770    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:36:59.415894 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:59.415872    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:36:59.416010 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:36:59.415987    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:00.416353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.415976    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:00.416921 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:00.416459    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:00.670907 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.670873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q49nn" event={"ID":"bd2f75cd-6470-4013-b544-d9f2b9ffdc37","Type":"ContainerStarted","Data":"e657f860178ef44b50add5afbdf02c7e318ea27c1dc1c71f031e153f104cee3b"}
Jul 19 14:37:00.672453 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.672427    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5t7hb" event={"ID":"ed8add3f-5454-4a1e-be52-f3131d364a2d","Type":"ContainerStarted","Data":"5ba846df3777fe3ca7d93929a4f95cd81475d95ba26ed028cb1bdd1e8dada716"}
Jul 19 14:37:00.674047 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.674019    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hm8sq" event={"ID":"4957289c-6405-4f3f-8531-5099ea2163ae","Type":"ContainerStarted","Data":"925a71a90990c2ce36eb7318cc5674e3ca32295bec5117d6130b5e50c668f7ed"}
Jul 19 14:37:00.675555 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.675533    2585 generic.go:356] "Generic (PLEG): container finished" podID="666bf80ba1e1983d2d3dd851bc8ab130" containerID="9c224b5ccb612521778411a40576c1df2388be5298c06c2712075f012328f07c" exitCode=0
Jul 19 14:37:00.675654 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.675598    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerDied","Data":"9c224b5ccb612521778411a40576c1df2388be5298c06c2712075f012328f07c"}
Jul 19 14:37:00.677038 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.677017    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" event={"ID":"3d37fd728dd468b7ee1c820042e75b2d","Type":"ContainerStarted","Data":"cc070058a4b3a5799df10ea2fd0e5e18fc65cce9c991ecfc163a8a2e7ceae5b4"}
Jul 19 14:37:00.678426 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.678404    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-556b6" event={"ID":"3807bf89-4f35-43e9-b170-b252aed8545a","Type":"ContainerStarted","Data":"8f2da9189a5cf7d62105d50de6a4f7a8325ca006d673288ccc39765df8fc0c90"}
Jul 19 14:37:00.680891 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.680876    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:37:00.681188 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681169    2585 generic.go:356] "Generic (PLEG): container finished" podID="6c96766b-2429-4fa1-ae33-92005de587c2" containerID="65ba2369067410d8313fcaf6cce18d2b85fd888571562a67e6ba4f2ed00070c6" exitCode=1
Jul 19 14:37:00.681265 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"69b2ff9ff15abd5fd2d9f5355a4ec73564501cf4a6d01f04b844f78db67dd8ad"}
Jul 19 14:37:00.681320 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681305    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"848cde313718083541edf66664b50aa22ca90d8f64d73dc9641895e130a360a5"}
Jul 19 14:37:00.681376 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681362    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"238c19c8c09ca9e76c94a224b7af8039e6052643f2e782e54607abfb10868ddf"}
Jul 19 14:37:00.681460 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681448    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"a0b0fa0582acc29188650d55eea761322db4f0382e5166c208c2f5ce435b510e"}
Jul 19 14:37:00.681500 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681482    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerDied","Data":"65ba2369067410d8313fcaf6cce18d2b85fd888571562a67e6ba4f2ed00070c6"}
Jul 19 14:37:00.681545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.681512    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"cb4a2537c554454da266e9eee6b03cf641bc59c3f891ea5c4a5f5b3075703ca3"}
Jul 19 14:37:00.682640 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.682621    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="f43fb6cbbbcf72df7d2c7a0ec9bda292d8add582751faaddb33587fa8910f354" exitCode=0
Jul 19 14:37:00.682696 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.682685    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"f43fb6cbbbcf72df7d2c7a0ec9bda292d8add582751faaddb33587fa8910f354"}
Jul 19 14:37:00.684144 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.684126    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5" event={"ID":"c066d4f2-b0cf-4a09-8306-901378c8ac79","Type":"ContainerStarted","Data":"463b02fdde3ffd444069050d6341695d123b22b90d18b91f9d6bae53805e0496"}
Jul 19 14:37:00.685317 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.685300    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-48n64" event={"ID":"cb6b1cf3-bddb-4991-a759-11fb984504be","Type":"ContainerStarted","Data":"92b74ff8a3537dc55cadea062dd1948303e685fa4bf8a39dc3f815a75837eb7f"}
Jul 19 14:37:00.700015 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.699969    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q49nn" podStartSLOduration=2.7263143899999998 podStartE2EDuration="22.699952266s" totalImagesPullingTime="19.973637876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.687298765 +0000 UTC m=+23.113775778" watchObservedRunningTime="2026-07-19 14:37:00.699952266 +0000 UTC m=+23.126429262"
Jul 19 14:37:00.700478 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.700457    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5t7hb" podStartSLOduration=2.817658109 podStartE2EDuration="22.700452191s" totalImagesPullingTime="19.882794082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.700022105 +0000 UTC m=+23.126499117" watchObservedRunningTime="2026-07-19 14:37:00.700452191 +0000 UTC m=+23.126929204"
Jul 19 14:37:00.732357 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.732264    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-556b6" podStartSLOduration=3.063709562 podStartE2EDuration="22.73224219s" totalImagesPullingTime="19.668532628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.715957294 +0000 UTC m=+23.142434308" watchObservedRunningTime="2026-07-19 14:37:00.73224219 +0000 UTC m=+23.158719203"
Jul 19 14:37:00.755740 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.755679    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hm8sq" podStartSLOduration=7.21303653 podStartE2EDuration="22.755658345s" totalImagesPullingTime="15.542621815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.732477356 +0000 UTC m=+23.158954362" watchObservedRunningTime="2026-07-19 14:37:00.755658345 +0000 UTC m=+23.182135377"
Jul 19 14:37:00.785270 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.785222    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-105.ec2.internal" podStartSLOduration=22.785203334 podStartE2EDuration="22.785203334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.784917748 +0000 UTC m=+23.211394762" watchObservedRunningTime="2026-07-19 14:37:00.785203334 +0000 UTC m=+23.211680349"
Jul 19 14:37:00.798277 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:00.798222    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-48n64" podStartSLOduration=2.8076511159999997 podStartE2EDuration="22.79820283s" totalImagesPullingTime="19.990551714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:00.797759856 +0000 UTC m=+23.224236870" watchObservedRunningTime="2026-07-19 14:37:00.79820283 +0000 UTC m=+23.224679844"
Jul 19 14:37:01.416086 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.416047    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:01.416358 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.416047    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:01.416358 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:01.416180    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:01.416358 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:01.416253    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:01.636898 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.636863    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 19 14:37:01.689692 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.689597    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5" event={"ID":"c066d4f2-b0cf-4a09-8306-901378c8ac79","Type":"ContainerStarted","Data":"1db5a8f2d8a0c62698045bb88f0224765aa8d2931720e84ace3f642f35637906"}
Jul 19 14:37:01.691695 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.691659    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" event={"ID":"666bf80ba1e1983d2d3dd851bc8ab130","Type":"ContainerStarted","Data":"89640fb542ec603960220bd20f909dd4ab2b0cd4e2fd2380dd9631ab2bbd0bba"}
Jul 19 14:37:01.693907 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.693881    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjnrw" event={"ID":"decdacd9-6e79-40b5-8cb5-32217e009188","Type":"ContainerStarted","Data":"e78a2ed55ec1ba91122901411f667f2671c4de0ed7cbf654987962c75ceb3ff2"}
Jul 19 14:37:01.705172 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.705119    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-105.ec2.internal" podStartSLOduration=23.705098413 podStartE2EDuration="23.705098413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:01.705063642 +0000 UTC m=+24.131540657" watchObservedRunningTime="2026-07-19 14:37:01.705098413 +0000 UTC m=+24.131575427"
Jul 19 14:37:01.718940 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:01.718888    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cjnrw" podStartSLOduration=4.1929969719999995 podStartE2EDuration="23.71886967s" totalImagesPullingTime="19.525872698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:01.71829913 +0000 UTC m=+24.144776144" watchObservedRunningTime="2026-07-19 14:37:01.71886967 +0000 UTC m=+24.145346690"
Jul 19 14:37:02.283567 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.283410    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-19T14:37:01.636888517Z","UUID":"6d4f287e-077f-4c62-9e72-649cc7849b05","Handler":null,"Name":"","Endpoint":""}
Jul 19 14:37:02.285382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.285357    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 19 14:37:02.285525 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.285406    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 19 14:37:02.416990 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.416853    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:02.417492 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:02.417000    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:02.700092 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.700064    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:37:02.700814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.700681    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"5846375a4d7512e0e7859ff8f8e96a049195e54a494d8507f73c0f302d5a4530"}
Jul 19 14:37:02.981911 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.981872    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:37:02.982675 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:02.982648    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:37:03.416595 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.416560    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:03.416814 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.416611    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:03.416814 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:03.416719    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:03.416920 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:03.416828    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:03.705129 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.705032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5" event={"ID":"c066d4f2-b0cf-4a09-8306-901378c8ac79","Type":"ContainerStarted","Data":"8c3bc105e29fc231c538cee4dfd7646f71b1b022b238ec0bdfc41b7170c68a17"}
Jul 19 14:37:03.705576 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.705280    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:37:03.705891 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.705872    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5t7hb"
Jul 19 14:37:03.721949 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.721888    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dtd5" podStartSLOduration=3.486519013 podStartE2EDuration="25.721864734s" totalImagesPullingTime="22.235345721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:03.72027321 +0000 UTC m=+26.146750221" watchObservedRunningTime="2026-07-19 14:37:03.721864734 +0000 UTC m=+26.148341748"
Jul 19 14:37:03.807574 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.807533    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:03.881655 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.881597    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:03.982663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:03.982572    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:04.032249 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:04.032208    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:04.416070 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:04.415994    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:04.416221 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:04.416110    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:05.408710 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.408507    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:05.416606 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.416578    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:05.416698 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.416616    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:05.416753 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:05.416705    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:05.416864 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:05.416840    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:05.481005 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.480969    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:05.713928 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.713841    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:37:05.714281 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.714254    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"72e8b3636b34bec6fbbc1a1cb7a2428c4e76a8c4cae5678118e50622545d0f2e"}
Jul 19 14:37:05.714490 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.714476    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:05.714725 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.714707    2585 scope.go:117] "RemoveContainer" containerID="65ba2369067410d8313fcaf6cce18d2b85fd888571562a67e6ba4f2ed00070c6"
Jul 19 14:37:05.716236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.716214    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="76cf6c5494200d8dc0813cd1324fd0f7b13d409dbaf4ea1c01ccdeb36407846b" exitCode=0
Jul 19 14:37:05.716336 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.716282    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"76cf6c5494200d8dc0813cd1324fd0f7b13d409dbaf4ea1c01ccdeb36407846b"}
Jul 19 14:37:05.732330 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:05.732306    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:06.419276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.419029    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:06.419804 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:06.419373    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:06.722215 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.722191    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:37:06.722587 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.722557    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" event={"ID":"6c96766b-2429-4fa1-ae33-92005de587c2","Type":"ContainerStarted","Data":"d4187413bf38c69ab272157cee67bf4e4e1edb851a4649bd688b3802dd677190"}
Jul 19 14:37:06.722770 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.722752    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:06.722858 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.722847    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:06.739077 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.739055    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:06.747559 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:06.747519    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt" podStartSLOduration=8.760292649 podStartE2EDuration="28.747502943s" totalImagesPullingTime="19.987210294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:06.746033704 +0000 UTC m=+29.172510717" watchObservedRunningTime="2026-07-19 14:37:06.747502943 +0000 UTC m=+29.173979955"
Jul 19 14:37:07.120893 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.120814    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57tbc"]
Jul 19 14:37:07.121063 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.120969    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:07.121101 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:07.121053    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:07.124496 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.124467    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jjgj4"]
Jul 19 14:37:07.124641 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.124581    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:07.124698 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:07.124675    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:07.125134 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.125109    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vftg8"]
Jul 19 14:37:07.125279 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.125200    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:07.125342 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:07.125278    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:07.134284 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.134258    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:07.233001 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.232953    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:07.726838 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.726808    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="9758c20d092800e9a84be418f7dfe7f34728308ffa53274a29da9024e1e5a30f" exitCode=0
Jul 19 14:37:07.727281 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:07.726893    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"9758c20d092800e9a84be418f7dfe7f34728308ffa53274a29da9024e1e5a30f"}
Jul 19 14:37:08.284650 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:08.284610    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:08.336716 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:08.336681    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:08.416659 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:08.416624    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:08.416911 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:08.416736    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:08.416911 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:08.416813    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:08.417055 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:08.416908    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:09.416698 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:09.416659    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:09.417097 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:09.416775    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:09.708094 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:09.708003    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:09.734955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:09.734925    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="40fefd368e7a0b0a1c0125563946a2a8fdb0a34bfc8f1093fc2fbab6313dac45" exitCode=0
Jul 19 14:37:09.735109 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:09.734971    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"40fefd368e7a0b0a1c0125563946a2a8fdb0a34bfc8f1093fc2fbab6313dac45"}
Jul 19 14:37:09.764031 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:09.763990    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:10.416404 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:10.416209    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:10.416609 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:10.416274    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:10.416609 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.416493    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:10.416609 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.416564    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:10.855768 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:10.855729    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:10.856250 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:10.855803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:10.856250 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.855886    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:37:10.856250 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.855911    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:37:10.856250 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.855958    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:37:42.855939439 +0000 UTC m=+65.282416444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:37:10.856250 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:10.855974    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret podName:d62cd493-14c5-4c1c-aef3-ce86ea253b26 nodeName:}" failed. No retries permitted until 2026-07-19 14:37:42.855965657 +0000 UTC m=+65.282442652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret") pod "global-pull-secret-syncer-jjgj4" (UID: "d62cd493-14c5-4c1c-aef3-ce86ea253b26") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:37:11.057729 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:11.057639    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:11.057917 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:11.057822    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:37:11.057917 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:11.057844    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:37:11.057917 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:11.057857    2585 projected.go:196] Error preparing data for projected volume kube-api-access-xdqk2 for pod openshift-network-diagnostics/network-check-target-57tbc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:37:11.058070 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:11.057919    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2 podName:ec4e78f5-0079-4759-bf05-fbac22c54822 nodeName:}" failed. No retries permitted until 2026-07-19 14:37:43.057902539 +0000 UTC m=+65.484379551 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xdqk2" (UniqueName: "kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2") pod "network-check-target-57tbc" (UID: "ec4e78f5-0079-4759-bf05-fbac22c54822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:37:11.416582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:11.416495    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:11.416740 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:11.416648    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:37:12.416937 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:12.416903    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:12.417450 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:12.417045    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57tbc" podUID="ec4e78f5-0079-4759-bf05-fbac22c54822"
Jul 19 14:37:12.417450 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:12.417294    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:12.417450 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:12.417384    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jjgj4" podUID="d62cd493-14c5-4c1c-aef3-ce86ea253b26"
Jul 19 14:37:12.944670 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:12.944638    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-105.ec2.internal" event="NodeReady"
Jul 19 14:37:12.944874 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:12.944822    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 19 14:37:12.982842 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:12.982807    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8lfqd"]
Jul 19 14:37:13.017601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.017564    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q6cqn"]
Jul 19 14:37:13.017765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.017640    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.021522 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.021498    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8vqq7\""
Jul 19 14:37:13.021902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.021881    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 19 14:37:13.022355 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.022337    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 19 14:37:13.033495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.033468    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8lfqd"]
Jul 19 14:37:13.033495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.033491    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.033664 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.033542    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6cqn"]
Jul 19 14:37:13.036425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.036289    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 19 14:37:13.036425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.036314    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 19 14:37:13.036425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.036316    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rbc88\""
Jul 19 14:37:13.036756 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.036736    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 19 14:37:13.073002 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.072957    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg8t8\" (UniqueName: \"kubernetes.io/projected/9565d067-376f-42c4-8fc2-81d72b2898e3-kube-api-access-pg8t8\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.073002 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.073003    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca2399d4-f674-4415-aade-10149711f91c-config-volume\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.073246 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.073032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2399d4-f674-4415-aade-10149711f91c-tmp-dir\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.073246 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.073088    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9565d067-376f-42c4-8fc2-81d72b2898e3-cert\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.073246 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.073110    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.073246 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.073187    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjfd4\" (UniqueName: \"kubernetes.io/projected/ca2399d4-f674-4415-aade-10149711f91c-kube-api-access-qjfd4\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.174405 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174363    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9565d067-376f-42c4-8fc2-81d72b2898e3-cert\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.174405 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174404    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174443    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjfd4\" (UniqueName: \"kubernetes.io/projected/ca2399d4-f674-4415-aade-10149711f91c-kube-api-access-qjfd4\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174533    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg8t8\" (UniqueName: \"kubernetes.io/projected/9565d067-376f-42c4-8fc2-81d72b2898e3-kube-api-access-pg8t8\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:13.174541    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174556    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca2399d4-f674-4415-aade-10149711f91c-config-volume\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174577    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2399d4-f674-4415-aade-10149711f91c-tmp-dir\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.174656 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:13.174617    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:13.674596433 +0000 UTC m=+36.101073435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:13.174981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.174927    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca2399d4-f674-4415-aade-10149711f91c-tmp-dir\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.175205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.175185    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca2399d4-f674-4415-aade-10149711f91c-config-volume\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.179605 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.179584    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9565d067-376f-42c4-8fc2-81d72b2898e3-cert\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.182948 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.182927    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg8t8\" (UniqueName: \"kubernetes.io/projected/9565d067-376f-42c4-8fc2-81d72b2898e3-kube-api-access-pg8t8\") pod \"ingress-canary-q6cqn\" (UID: \"9565d067-376f-42c4-8fc2-81d72b2898e3\") " pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.183079 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.182995    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjfd4\" (UniqueName: \"kubernetes.io/projected/ca2399d4-f674-4415-aade-10149711f91c-kube-api-access-qjfd4\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.342959 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.342917    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6cqn"
Jul 19 14:37:13.416436 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.416399    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:13.419016 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.418972    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f5r6f\""
Jul 19 14:37:13.419673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.419026    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:37:13.679464 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.679381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:13.679623 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:13.679568    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:13.679666 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:13.679652    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:14.679627568 +0000 UTC m=+37.106104566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:13.987469 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:13.987436    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6cqn"]
Jul 19 14:37:14.054090 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:37:14.054052    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9565d067_376f_42c4_8fc2_81d72b2898e3.slice/crio-820ee47debc2755f135ee5de7cbfce86ea4afa38117d961658241594014db737 WatchSource:0}: Error finding container 820ee47debc2755f135ee5de7cbfce86ea4afa38117d961658241594014db737: Status 404 returned error can't find the container with id 820ee47debc2755f135ee5de7cbfce86ea4afa38117d961658241594014db737
Jul 19 14:37:14.417027 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.416834    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:14.417198 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.416853    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:14.419462 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.419436    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:37:14.420051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.419490    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:37:14.420649 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.420549    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:37:14.420649 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.420608    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xgfj\""
Jul 19 14:37:14.686252 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.686157    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:14.686475 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:14.686281    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:14.686475 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:14.686356    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:16.686334857 +0000 UTC m=+39.112811852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:14.749066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.749018    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6cqn" event={"ID":"9565d067-376f-42c4-8fc2-81d72b2898e3","Type":"ContainerStarted","Data":"820ee47debc2755f135ee5de7cbfce86ea4afa38117d961658241594014db737"}
Jul 19 14:37:14.752224 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.752193    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="7fdcc9ce9e3703aad46d9f1139b19df32a5c2b51680f13c42116d6a9f32d1adf" exitCode=0
Jul 19 14:37:14.752360 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:14.752239    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"7fdcc9ce9e3703aad46d9f1139b19df32a5c2b51680f13c42116d6a9f32d1adf"}
Jul 19 14:37:15.758524 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:15.758490    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92bec5f-8091-47b8-bfcd-f0d15fec6559" containerID="884d710a46aa2b0db00119627c0f7eb63580ddf83ba27545fc979bbd0b62751f" exitCode=0
Jul 19 14:37:15.758941 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:15.758568    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerDied","Data":"884d710a46aa2b0db00119627c0f7eb63580ddf83ba27545fc979bbd0b62751f"}
Jul 19 14:37:16.702487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.702441    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:16.702672 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:16.702643    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:16.702749 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:16.702738    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:20.702716214 +0000 UTC m=+43.129193208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:16.763841 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.763800    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" event={"ID":"b92bec5f-8091-47b8-bfcd-f0d15fec6559","Type":"ContainerStarted","Data":"a77e7c57cb92eb0e116bbd1ac17052b893d77ed344ea5421f12137ef99ae6583"}
Jul 19 14:37:16.765297 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.765261    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6cqn" event={"ID":"9565d067-376f-42c4-8fc2-81d72b2898e3","Type":"ContainerStarted","Data":"bd2721a2fd726b124a3d28eea3870790c24aa90a73f9f1c9a7dff11ab5b6be74"}
Jul 19 14:37:16.784541 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.784483    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h4cnb" podStartSLOduration=8.037258717 podStartE2EDuration="38.784465571s" totalImagesPullingTime="30.747206854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:16.783548389 +0000 UTC m=+39.210025402" watchObservedRunningTime="2026-07-19 14:37:16.784465571 +0000 UTC m=+39.210942578"
Jul 19 14:37:16.797844 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.797771    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q6cqn" podStartSLOduration=2.385887833 podStartE2EDuration="4.79775377s" totalImagesPullingTime="2.411865937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:16.796922559 +0000 UTC m=+39.223399573" watchObservedRunningTime="2026-07-19 14:37:16.79775377 +0000 UTC m=+39.224230801"
Jul 19 14:37:16.872324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.872231    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:16.930339 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:16.930292    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:17.963127 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:17.963090    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:20.736223 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:20.736189    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:20.736626 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:20.736298    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:20.736626 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:20.736356    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:28.736339492 +0000 UTC m=+51.162816484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:28.794082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:28.794045    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:28.794549 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:28.794199    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:28.794549 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:28.794276    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:37:44.794258685 +0000 UTC m=+67.220735679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:31.608390 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:31.608351    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:31.662628 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:31.662589    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:38.740486 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:38.740456    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrwlt"
Jul 19 14:37:38.869251 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:38.869215    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:38.918081 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:38.918043    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:38.966393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:38.966359    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:42.114906 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.114866    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:37:42.903487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.903432    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:37:42.903675 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.903510    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:42.905902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.905882    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:37:42.905976 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.905903    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:37:42.914685 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:42.914662    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:37:42.914776 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:42.914756    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:38:46.914730887 +0000 UTC m=+129.341207890 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : secret "metrics-daemon-secret" not found
Jul 19 14:37:42.918651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.918628    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d62cd493-14c5-4c1c-aef3-ce86ea253b26-original-pull-secret\") pod \"global-pull-secret-syncer-jjgj4\" (UID: \"d62cd493-14c5-4c1c-aef3-ce86ea253b26\") " pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:42.929283 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:42.929262    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jjgj4"
Jul 19 14:37:43.061055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.061022    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jjgj4"]
Jul 19 14:37:43.064233 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:37:43.064200    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62cd493_14c5_4c1c_aef3_ce86ea253b26.slice/crio-e3f433446a2271acd9599d70a50b834fbe2334c24aaec7df2bebedcf05230bfd WatchSource:0}: Error finding container e3f433446a2271acd9599d70a50b834fbe2334c24aaec7df2bebedcf05230bfd: Status 404 returned error can't find the container with id e3f433446a2271acd9599d70a50b834fbe2334c24aaec7df2bebedcf05230bfd
Jul 19 14:37:43.105774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.105737    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:43.108472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.108450    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:37:43.117958 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.117939    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:37:43.129080 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.129056    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqk2\" (UniqueName: \"kubernetes.io/projected/ec4e78f5-0079-4759-bf05-fbac22c54822-kube-api-access-xdqk2\") pod \"network-check-target-57tbc\" (UID: \"ec4e78f5-0079-4759-bf05-fbac22c54822\") " pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:43.236988 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.236966    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xgfj\""
Jul 19 14:37:43.244858 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.244820    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:43.370887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.370856    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57tbc"]
Jul 19 14:37:43.374284 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:37:43.374259    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4e78f5_0079_4759_bf05_fbac22c54822.slice/crio-717eaeb4b06f85ee07af3e05275b629758a52cf8f8022a8d049eddac40014dbc WatchSource:0}: Error finding container 717eaeb4b06f85ee07af3e05275b629758a52cf8f8022a8d049eddac40014dbc: Status 404 returned error can't find the container with id 717eaeb4b06f85ee07af3e05275b629758a52cf8f8022a8d049eddac40014dbc
Jul 19 14:37:43.828473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.828429    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57tbc" event={"ID":"ec4e78f5-0079-4759-bf05-fbac22c54822","Type":"ContainerStarted","Data":"717eaeb4b06f85ee07af3e05275b629758a52cf8f8022a8d049eddac40014dbc"}
Jul 19 14:37:43.829578 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:43.829549    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jjgj4" event={"ID":"d62cd493-14c5-4c1c-aef3-ce86ea253b26","Type":"ContainerStarted","Data":"e3f433446a2271acd9599d70a50b834fbe2334c24aaec7df2bebedcf05230bfd"}
Jul 19 14:37:44.818770 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:44.818726    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:37:44.819262 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:44.818890    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:37:44.819262 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:37:44.818963    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:38:16.818942171 +0000 UTC m=+99.245419167 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:37:48.843581 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:48.843534    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jjgj4" event={"ID":"d62cd493-14c5-4c1c-aef3-ce86ea253b26","Type":"ContainerStarted","Data":"a461251cbcfa6b3887c518920208e1a6abf4fc6ae67c596f4517312fc637f12d"}
Jul 19 14:37:48.844887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:48.844865    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57tbc" event={"ID":"ec4e78f5-0079-4759-bf05-fbac22c54822","Type":"ContainerStarted","Data":"271875fb4a95cf984fb5024c77f41a4939d28aa6a0da119ad3e05131b0b8d112"}
Jul 19 14:37:48.844998 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:48.844987    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:37:48.857380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:48.857330    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jjgj4" podStartSLOduration=65.773918552 podStartE2EDuration="1m10.857316069s" totalImagesPullingTime="5.083397517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:48.85667082 +0000 UTC m=+71.283147833" watchObservedRunningTime="2026-07-19 14:37:48.857316069 +0000 UTC m=+71.283793081"
Jul 19 14:37:48.869987 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:37:48.869939    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-57tbc" podStartSLOduration=66.087794406 podStartE2EDuration="1m10.869929326s" totalImagesPullingTime="4.78213492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:37:48.869246094 +0000 UTC m=+71.295723106" watchObservedRunningTime="2026-07-19 14:37:48.869929326 +0000 UTC m=+71.296406338"
Jul 19 14:38:11.989393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:11.989226    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:12.036733 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:12.036685    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:38:16.832995 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:16.832960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:38:16.833403 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:38:16.833125    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:38:16.833403 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:38:16.833198    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:39:20.833181671 +0000 UTC m=+163.259658662 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:38:19.849768 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:19.849739    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-57tbc"
Jul 19 14:38:28.096363 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.096328    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"]
Jul 19 14:38:28.099357 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.099316    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.099799 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.099756    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"]
Jul 19 14:38:28.104007 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.102807    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.104343 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.104311    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 19 14:38:28.104605 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.104589    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 19 14:38:28.104678 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.104635    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4mjqv\""
Jul 19 14:38:28.104727 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.104635    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 19 14:38:28.104800 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.104704    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 19 14:38:28.105267 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.105253    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 19 14:38:28.110040 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.110019    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"]
Jul 19 14:38:28.112848 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.112825    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"]
Jul 19 14:38:28.208533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.208492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-tmp\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.208533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.208534    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2cbe3dd-9926-4fee-b55a-d5a417b50090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.208812 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.208551    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77j5\" (UniqueName: \"kubernetes.io/projected/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-kube-api-access-d77j5\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.208812 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.208571    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.208812 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.208665    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szxfp\" (UniqueName: \"kubernetes.io/projected/c2cbe3dd-9926-4fee-b55a-d5a417b50090-kube-api-access-szxfp\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.309458 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.309419    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2cbe3dd-9926-4fee-b55a-d5a417b50090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.309458 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.309458    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77j5\" (UniqueName: \"kubernetes.io/projected/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-kube-api-access-d77j5\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.309721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.309478    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.309721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.309533    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szxfp\" (UniqueName: \"kubernetes.io/projected/c2cbe3dd-9926-4fee-b55a-d5a417b50090-kube-api-access-szxfp\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.309721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.309672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-tmp\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.310074 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.310044    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-tmp\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.311981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.311959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2cbe3dd-9926-4fee-b55a-d5a417b50090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.312103 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.311987    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.317506 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.317482    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szxfp\" (UniqueName: \"kubernetes.io/projected/c2cbe3dd-9926-4fee-b55a-d5a417b50090-kube-api-access-szxfp\") pod \"managed-serviceaccount-addon-agent-798fc68898-wn6bj\" (UID: \"c2cbe3dd-9926-4fee-b55a-d5a417b50090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.317631 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.317617    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77j5\" (UniqueName: \"kubernetes.io/projected/e6d76a65-8105-4e58-bf45-19fb2fec2c1d-kube-api-access-d77j5\") pod \"klusterlet-addon-workmgr-67c6ffd45d-tqjmt\" (UID: \"e6d76a65-8105-4e58-bf45-19fb2fec2c1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.418227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.418127    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"
Jul 19 14:38:28.423423 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.423399    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:28.551746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.551582    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj"]
Jul 19 14:38:28.554584 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:38:28.554548    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2cbe3dd_9926_4fee_b55a_d5a417b50090.slice/crio-85f79929975c0a7018b64655e9f5b295be7e66a7590a11cd56eda5729b45edd6 WatchSource:0}: Error finding container 85f79929975c0a7018b64655e9f5b295be7e66a7590a11cd56eda5729b45edd6: Status 404 returned error can't find the container with id 85f79929975c0a7018b64655e9f5b295be7e66a7590a11cd56eda5729b45edd6
Jul 19 14:38:28.571413 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.571377    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"]
Jul 19 14:38:28.577035 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:38:28.576999    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d76a65_8105_4e58_bf45_19fb2fec2c1d.slice/crio-d666a7dc9c3331e9b40223c94854b972c612c8bef55774ce7c802b62d217d5fc WatchSource:0}: Error finding container d666a7dc9c3331e9b40223c94854b972c612c8bef55774ce7c802b62d217d5fc: Status 404 returned error can't find the container with id d666a7dc9c3331e9b40223c94854b972c612c8bef55774ce7c802b62d217d5fc
Jul 19 14:38:28.942983 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.942947    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" event={"ID":"e6d76a65-8105-4e58-bf45-19fb2fec2c1d","Type":"ContainerStarted","Data":"d666a7dc9c3331e9b40223c94854b972c612c8bef55774ce7c802b62d217d5fc"}
Jul 19 14:38:28.943951 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:28.943927    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj" event={"ID":"c2cbe3dd-9926-4fee-b55a-d5a417b50090","Type":"ContainerStarted","Data":"85f79929975c0a7018b64655e9f5b295be7e66a7590a11cd56eda5729b45edd6"}
Jul 19 14:38:32.958378 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:32.958339    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" event={"ID":"e6d76a65-8105-4e58-bf45-19fb2fec2c1d","Type":"ContainerStarted","Data":"5b2afba5c377c90acdae94ac9437741ef85be23fdd608750b1ebba41b87845a5"}
Jul 19 14:38:32.958883 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:32.958534    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:32.960046 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:32.960005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj" event={"ID":"c2cbe3dd-9926-4fee-b55a-d5a417b50090","Type":"ContainerStarted","Data":"5834af6360f5dd8dfd6f51d8eac46a65eee6ec59ede00f7449c5b5a9711d030a"}
Jul 19 14:38:32.960429 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:32.960413    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:38:32.976389 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:32.976339    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" podStartSLOduration=1.09622577 podStartE2EDuration="4.976325064s" totalImagesPullingTime="3.880099294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:38:32.975305871 +0000 UTC m=+115.401782883" watchObservedRunningTime="2026-07-19 14:38:32.976325064 +0000 UTC m=+115.402802079"
Jul 19 14:38:33.002446 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:33.002394    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj" podStartSLOduration=1.109791331 podStartE2EDuration="5.002379991s" totalImagesPullingTime="3.89258866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:38:33.001871454 +0000 UTC m=+115.428348470" watchObservedRunningTime="2026-07-19 14:38:33.002379991 +0000 UTC m=+115.428857004"
Jul 19 14:38:46.939724 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:46.939669    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:38:46.940178 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:38:46.939847    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:38:46.940178 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:38:46.939919    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:40:48.939902321 +0000 UTC m=+251.366379312 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : secret "metrics-daemon-secret" not found
Jul 19 14:38:57.601591 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:38:57.601553    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:39:16.029571 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:39:16.029502    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8lfqd" podUID="ca2399d4-f674-4415-aade-10149711f91c"
Jul 19 14:39:16.080076 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:39:16.080047    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:39:16.427654 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:39:16.427557    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:39:20.888070 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:39:20.888036    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:39:20.888472 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:39:20.888159    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:39:20.888472 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:39:20.888223    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls podName:ca2399d4-f674-4415-aade-10149711f91c nodeName:}" failed. No retries permitted until 2026-07-19 14:41:22.888208941 +0000 UTC m=+285.314685932 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls") pod "dns-default-8lfqd" (UID: "ca2399d4-f674-4415-aade-10149711f91c") : secret "dns-default-metrics-tls" not found
Jul 19 14:39:28.417475 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:39:28.417434    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:40:32.959208 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:32.959102    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" podUID="e6d76a65-8105-4e58-bf45-19fb2fec2c1d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.9:8000/readyz\": dial tcp 10.134.0.9:8000: connect: connection refused"
Jul 19 14:40:33.293487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.293454    2585 generic.go:356] "Generic (PLEG): container finished" podID="e6d76a65-8105-4e58-bf45-19fb2fec2c1d" containerID="5b2afba5c377c90acdae94ac9437741ef85be23fdd608750b1ebba41b87845a5" exitCode=1
Jul 19 14:40:33.293670 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.293532    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" event={"ID":"e6d76a65-8105-4e58-bf45-19fb2fec2c1d","Type":"ContainerDied","Data":"5b2afba5c377c90acdae94ac9437741ef85be23fdd608750b1ebba41b87845a5"}
Jul 19 14:40:33.293936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.293909    2585 scope.go:117] "RemoveContainer" containerID="5b2afba5c377c90acdae94ac9437741ef85be23fdd608750b1ebba41b87845a5"
Jul 19 14:40:33.295068 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.294998    2585 generic.go:356] "Generic (PLEG): container finished" podID="c2cbe3dd-9926-4fee-b55a-d5a417b50090" containerID="5834af6360f5dd8dfd6f51d8eac46a65eee6ec59ede00f7449c5b5a9711d030a" exitCode=255
Jul 19 14:40:33.295149 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.295066    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj" event={"ID":"c2cbe3dd-9926-4fee-b55a-d5a417b50090","Type":"ContainerDied","Data":"5834af6360f5dd8dfd6f51d8eac46a65eee6ec59ede00f7449c5b5a9711d030a"}
Jul 19 14:40:33.300809 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:33.300795    2585 scope.go:117] "RemoveContainer" containerID="5834af6360f5dd8dfd6f51d8eac46a65eee6ec59ede00f7449c5b5a9711d030a"
Jul 19 14:40:34.299197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:34.299165    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-798fc68898-wn6bj" event={"ID":"c2cbe3dd-9926-4fee-b55a-d5a417b50090","Type":"ContainerStarted","Data":"1bcc655ec70cbe81093235a0ed5ab7314fd0315e90bc0706f9c92653139dd1dc"}
Jul 19 14:40:34.300740 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:34.300716    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt" event={"ID":"e6d76a65-8105-4e58-bf45-19fb2fec2c1d","Type":"ContainerStarted","Data":"1c8a52b2475c403c364d7db1ca4118ed8bd7284748a1517513b62f70c87b66a5"}
Jul 19 14:40:34.300992 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:34.300975    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:40:34.301509 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:34.301490    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c6ffd45d-tqjmt"
Jul 19 14:40:48.996259 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:48.996208    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:40:48.996657 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:48.996367    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:40:48.996657 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:48.996434    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs podName:223f7083-66c8-4efb-9c2e-f19a4b4724be nodeName:}" failed. No retries permitted until 2026-07-19 14:42:50.996414733 +0000 UTC m=+373.422891727 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs") pod "network-metrics-daemon-vftg8" (UID: "223f7083-66c8-4efb-9c2e-f19a4b4724be") : secret "metrics-daemon-secret" not found
Jul 19 14:40:55.658831 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.658795    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"]
Jul 19 14:40:55.662014 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.661993    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"
Jul 19 14:40:55.668366 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.668340    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 19 14:40:55.668614 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.668598    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:40:55.671929 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.671910    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2t24h\""
Jul 19 14:40:55.679347 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.679324    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"]
Jul 19 14:40:55.744255 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.744221    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9lm\" (UniqueName: \"kubernetes.io/projected/8aafa04c-a08c-4dc2-bd74-8e916c016e45-kube-api-access-dm9lm\") pod \"volume-data-source-validator-78cf4c6f6-5zk42\" (UID: \"8aafa04c-a08c-4dc2-bd74-8e916c016e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"
Jul 19 14:40:55.747333 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.747300    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"]
Jul 19 14:40:55.750018 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.749995    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"]
Jul 19 14:40:55.750150 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.750134    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.752563 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.752539    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:40:55.752715 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.752535    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vm5gh\""
Jul 19 14:40:55.752715 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.752581    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.752715 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.752542    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 19 14:40:55.753015 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.752999    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 19 14:40:55.756927 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.756846    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:40:55.757086 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.757067    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 19 14:40:55.757165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.757124    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dbgcj\""
Jul 19 14:40:55.757294 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.757175    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 19 14:40:55.757400 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.757366    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 19 14:40:55.763840 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.763813    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"]
Jul 19 14:40:55.764883 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.764864    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"]
Jul 19 14:40:55.845144 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845103    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.845144 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845142    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f6f2\" (UniqueName: \"kubernetes.io/projected/49005db1-e3a4-446e-b0e8-2e3994ae649b-kube-api-access-9f6f2\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.845379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845184    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9lm\" (UniqueName: \"kubernetes.io/projected/8aafa04c-a08c-4dc2-bd74-8e916c016e45-kube-api-access-dm9lm\") pod \"volume-data-source-validator-78cf4c6f6-5zk42\" (UID: \"8aafa04c-a08c-4dc2-bd74-8e916c016e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"
Jul 19 14:40:55.845379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845207    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkmr5\" (UniqueName: \"kubernetes.io/projected/0805c402-8f62-4c36-9c47-9c66d82625da-kube-api-access-rkmr5\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.845379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845261    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0805c402-8f62-4c36-9c47-9c66d82625da-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.845379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.845283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0805c402-8f62-4c36-9c47-9c66d82625da-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.854327 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.854304    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9lm\" (UniqueName: \"kubernetes.io/projected/8aafa04c-a08c-4dc2-bd74-8e916c016e45-kube-api-access-dm9lm\") pod \"volume-data-source-validator-78cf4c6f6-5zk42\" (UID: \"8aafa04c-a08c-4dc2-bd74-8e916c016e45\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"
Jul 19 14:40:55.918435 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.918348    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:55.946074 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946024    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0805c402-8f62-4c36-9c47-9c66d82625da-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.946074 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946078    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0805c402-8f62-4c36-9c47-9c66d82625da-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.946316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946133    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.946316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946156    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f6f2\" (UniqueName: \"kubernetes.io/projected/49005db1-e3a4-446e-b0e8-2e3994ae649b-kube-api-access-9f6f2\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.946316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946192    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkmr5\" (UniqueName: \"kubernetes.io/projected/0805c402-8f62-4c36-9c47-9c66d82625da-kube-api-access-rkmr5\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.946316 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:55.946287    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 19 14:40:55.946519 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:55.946371    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls podName:49005db1-e3a4-446e-b0e8-2e3994ae649b nodeName:}" failed. No retries permitted until 2026-07-19 14:40:56.446350647 +0000 UTC m=+258.872827655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7l949" (UID: "49005db1-e3a4-446e-b0e8-2e3994ae649b") : secret "samples-operator-tls" not found
Jul 19 14:40:55.946737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.946717    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0805c402-8f62-4c36-9c47-9c66d82625da-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.948319 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.948296    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0805c402-8f62-4c36-9c47-9c66d82625da-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.954990 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.954968    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f6f2\" (UniqueName: \"kubernetes.io/projected/49005db1-e3a4-446e-b0e8-2e3994ae649b-kube-api-access-9f6f2\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:55.955129 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.955110    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkmr5\" (UniqueName: \"kubernetes.io/projected/0805c402-8f62-4c36-9c47-9c66d82625da-kube-api-access-rkmr5\") pod \"kube-storage-version-migrator-operator-7658d9b749-rzgsz\" (UID: \"0805c402-8f62-4c36-9c47-9c66d82625da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:55.971061 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:55.971018    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"
Jul 19 14:40:56.006429 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.006399    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:56.066163 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.066132    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"
Jul 19 14:40:56.112417 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.112384    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42"]
Jul 19 14:40:56.118004 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:40:56.117960    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aafa04c_a08c_4dc2_bd74_8e916c016e45.slice/crio-56aade18652d15c1735da8cf1966f43bd481f403460fac343419d9af4473da49 WatchSource:0}: Error finding container 56aade18652d15c1735da8cf1966f43bd481f403460fac343419d9af4473da49: Status 404 returned error can't find the container with id 56aade18652d15c1735da8cf1966f43bd481f403460fac343419d9af4473da49
Jul 19 14:40:56.193869 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.193776    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz"]
Jul 19 14:40:56.196386 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:40:56.196355    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0805c402_8f62_4c36_9c47_9c66d82625da.slice/crio-23055c7998bbf9b6ccda5c83111b3e2112393fe0e593ff705d46b7a51655a1fd WatchSource:0}: Error finding container 23055c7998bbf9b6ccda5c83111b3e2112393fe0e593ff705d46b7a51655a1fd: Status 404 returned error can't find the container with id 23055c7998bbf9b6ccda5c83111b3e2112393fe0e593ff705d46b7a51655a1fd
Jul 19 14:40:56.366824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.366769    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz" event={"ID":"0805c402-8f62-4c36-9c47-9c66d82625da","Type":"ContainerStarted","Data":"23055c7998bbf9b6ccda5c83111b3e2112393fe0e593ff705d46b7a51655a1fd"}
Jul 19 14:40:56.367718 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.367696    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42" event={"ID":"8aafa04c-a08c-4dc2-bd74-8e916c016e45","Type":"ContainerStarted","Data":"56aade18652d15c1735da8cf1966f43bd481f403460fac343419d9af4473da49"}
Jul 19 14:40:56.450696 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:56.450607    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:56.450890 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:56.450761    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 19 14:40:56.450890 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:56.450870    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls podName:49005db1-e3a4-446e-b0e8-2e3994ae649b nodeName:}" failed. No retries permitted until 2026-07-19 14:40:57.450852122 +0000 UTC m=+259.877329114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7l949" (UID: "49005db1-e3a4-446e-b0e8-2e3994ae649b") : secret "samples-operator-tls" not found
Jul 19 14:40:57.458822 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:57.458761    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:57.459296 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:57.458894    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 19 14:40:57.459296 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:57.458951    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls podName:49005db1-e3a4-446e-b0e8-2e3994ae649b nodeName:}" failed. No retries permitted until 2026-07-19 14:40:59.458936713 +0000 UTC m=+261.885413705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7l949" (UID: "49005db1-e3a4-446e-b0e8-2e3994ae649b") : secret "samples-operator-tls" not found
Jul 19 14:40:58.374334 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.374291    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz" event={"ID":"0805c402-8f62-4c36-9c47-9c66d82625da","Type":"ContainerStarted","Data":"8322fef1f30560839d07e741de0d3a793888b32249387509c726c985fa09857e"}
Jul 19 14:40:58.375834 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.375808    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42" event={"ID":"8aafa04c-a08c-4dc2-bd74-8e916c016e45","Type":"ContainerStarted","Data":"28d8fcd2fcadfdd6e6e15c71c5000daca1b6806e990d0250fd451400ab9541c5"}
Jul 19 14:40:58.390583 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.390526    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz" podStartSLOduration=1.427529464 podStartE2EDuration="3.39051199s" totalImagesPullingTime="1.962982526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:40:58.389668977 +0000 UTC m=+260.816145991" watchObservedRunningTime="2026-07-19 14:40:58.39051199 +0000 UTC m=+260.816989003"
Jul 19 14:40:58.404628 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.404563    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-5zk42" podStartSLOduration=2.155768669 podStartE2EDuration="3.404542294s" totalImagesPullingTime="1.248773625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:40:58.403687453 +0000 UTC m=+260.830164467" watchObservedRunningTime="2026-07-19 14:40:58.404542294 +0000 UTC m=+260.831019308"
Jul 19 14:40:58.485651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.485620    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:58.543018 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:58.542980    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:59.473459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.473418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:40:59.473683 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:59.473597    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 19 14:40:59.473683 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:40:59.473676    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls podName:49005db1-e3a4-446e-b0e8-2e3994ae649b nodeName:}" failed. No retries permitted until 2026-07-19 14:41:03.473654783 +0000 UTC m=+265.900131781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7l949" (UID: "49005db1-e3a4-446e-b0e8-2e3994ae649b") : secret "samples-operator-tls" not found
Jul 19 14:40:59.813380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.813338    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:59.826027 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.825958    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"]
Jul 19 14:40:59.828848 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.828824    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"
Jul 19 14:40:59.831326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.831303    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 19 14:40:59.831749 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.831730    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-28d4p\""
Jul 19 14:40:59.832192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.832174    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 19 14:40:59.840545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.840518    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"]
Jul 19 14:40:59.894059 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.894020    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:59.948709 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.948675    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:40:59.977000 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:40:59.976962    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnrw\" (UniqueName: \"kubernetes.io/projected/faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48-kube-api-access-5lnrw\") pod \"migrator-7cc9c7ffd7-h6x8z\" (UID: \"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"
Jul 19 14:41:00.077737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.077648    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnrw\" (UniqueName: \"kubernetes.io/projected/faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48-kube-api-access-5lnrw\") pod \"migrator-7cc9c7ffd7-h6x8z\" (UID: \"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"
Jul 19 14:41:00.086044 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.086012    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnrw\" (UniqueName: \"kubernetes.io/projected/faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48-kube-api-access-5lnrw\") pod \"migrator-7cc9c7ffd7-h6x8z\" (UID: \"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"
Jul 19 14:41:00.138255 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.138216    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"
Jul 19 14:41:00.189839 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.189809    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmgwr"]
Jul 19 14:41:00.193171 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.193147    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.195991 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.195761    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 19 14:41:00.195991 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.195838    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 19 14:41:00.195991 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.195760    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 19 14:41:00.196230 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.196214    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 19 14:41:00.196273 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.196262    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qjwr\""
Jul 19 14:41:00.206399 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.206318    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmgwr"]
Jul 19 14:41:00.214335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.214300    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:00.272048 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.272013    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z"]
Jul 19 14:41:00.275125 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:00.275096    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaedc00c_ea8b_4eb2_a4c6_7dc4e5f82a48.slice/crio-3cb2f784be3037bbd2a3c21fc2f9335793612f5c39fca95b1271869d716a3793 WatchSource:0}: Error finding container 3cb2f784be3037bbd2a3c21fc2f9335793612f5c39fca95b1271869d716a3793: Status 404 returned error can't find the container with id 3cb2f784be3037bbd2a3c21fc2f9335793612f5c39fca95b1271869d716a3793
Jul 19 14:41:00.278365 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.278338    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:00.279453 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.279422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcs94\" (UniqueName: \"kubernetes.io/projected/6530713d-747e-4cbe-a95b-f897147db574-kube-api-access-dcs94\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.279547 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.279487    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6530713d-747e-4cbe-a95b-f897147db574-crio-socket\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.279547 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.279511    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.279547 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.279529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.279666 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.279608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6530713d-747e-4cbe-a95b-f897147db574-data-volume\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380495    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6530713d-747e-4cbe-a95b-f897147db574-data-volume\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380588    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcs94\" (UniqueName: \"kubernetes.io/projected/6530713d-747e-4cbe-a95b-f897147db574-kube-api-access-dcs94\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6530713d-747e-4cbe-a95b-f897147db574-crio-socket\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380659    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380754    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6530713d-747e-4cbe-a95b-f897147db574-crio-socket\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.380808    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.380818    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.380858    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6530713d-747e-4cbe-a95b-f897147db574-data-volume\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.380887 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.380881    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:00.88086337 +0000 UTC m=+263.307340395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : configmap "kube-rbac-proxy" not found
Jul 19 14:41:00.381207 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.380913    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:00.880893276 +0000 UTC m=+263.307370282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:00.383994 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.383968    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z" event={"ID":"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48","Type":"ContainerStarted","Data":"3cb2f784be3037bbd2a3c21fc2f9335793612f5c39fca95b1271869d716a3793"}
Jul 19 14:41:00.388703 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.388680    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcs94\" (UniqueName: \"kubernetes.io/projected/6530713d-747e-4cbe-a95b-f897147db574-kube-api-access-dcs94\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.475259 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.475225    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:00.531033 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.530995    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:00.836718 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.836671    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:00.884625 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.884584    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.884625 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.884626    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:00.884894 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.884702    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:41:00.884894 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.884774    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:01.884755133 +0000 UTC m=+264.311232125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : configmap "kube-rbac-proxy" not found
Jul 19 14:41:00.884894 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.884799    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:00.884894 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:00.884847    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:01.884834165 +0000 UTC m=+264.311311156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:00.938020 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:00.937983    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:01.114546 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:01.114461    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:01.216442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:01.216403    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:01.893589 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:01.893480    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:01.893589 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:01.893523    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:01.894065 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:01.893585    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:41:01.894065 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:01.893611    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:01.894065 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:01.893662    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:03.89364813 +0000 UTC m=+266.320125121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:01.894065 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:01.893676    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:03.893669536 +0000 UTC m=+266.320146527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : configmap "kube-rbac-proxy" not found
Jul 19 14:41:02.395164 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.395128    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z" event={"ID":"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48","Type":"ContainerStarted","Data":"f509fe02fcdf987cdb723209d16fe140fc8892e9ef15a3d87a8e3eb1ef35ed85"}
Jul 19 14:41:02.395326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.395209    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z" event={"ID":"faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48","Type":"ContainerStarted","Data":"f0fd705752f33b77a3b43471d63f7e3fdab2f108eccbc20c731a0ad7adbe4ca7"}
Jul 19 14:41:02.410165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.410110    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-h6x8z" podStartSLOduration=2.12823093 podStartE2EDuration="3.410093245s" totalImagesPullingTime="1.281862315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:02.4091287 +0000 UTC m=+264.835605714" watchObservedRunningTime="2026-07-19 14:41:02.410093245 +0000 UTC m=+264.836570255"
Jul 19 14:41:02.479832 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.479796    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:02.536095 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.536061    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:02.755974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.755938    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:02.856741 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.856708    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:02.869422 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:02.869390    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-48n64_cb6b1cf3-bddb-4991-a759-11fb984504be/dns-node-resolver/0.log"
Jul 19 14:41:03.506578 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:03.506519    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:41:03.507209 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.506695    2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 19 14:41:03.507209 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.506774    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls podName:49005db1-e3a4-446e-b0e8-2e3994ae649b nodeName:}" failed. No retries permitted until 2026-07-19 14:41:11.506754254 +0000 UTC m=+273.933231267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7l949" (UID: "49005db1-e3a4-446e-b0e8-2e3994ae649b") : secret "samples-operator-tls" not found
Jul 19 14:41:03.731767 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:03.731730    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:03.909156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:03.909048    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:03.909156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:03.909093    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:03.909372 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.909175    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:41:03.909372 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.909206    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:03.909372 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.909246    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:07.909230704 +0000 UTC m=+270.335707696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : configmap "kube-rbac-proxy" not found
Jul 19 14:41:03.909372 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:03.909260    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:07.909254513 +0000 UTC m=+270.335731504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:04.069827 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.069778    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hm8sq_4957289c-6405-4f3f-8531-5099ea2163ae/node-ca/0.log"
Jul 19 14:41:04.408487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.408447    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:04.521353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.521320    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:04.692640 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.692560    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:04.792523 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.792479    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:04.854425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:04.854391    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:05.070523 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:05.070494    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6cqn_9565d067-376f-42c4-8fc2-81d72b2898e3/serve-healthcheck-canary/0.log"
Jul 19 14:41:05.105545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:05.105515    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:05.270192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:05.270160    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-h6x8z_faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48/migrator/0.log"
Jul 19 14:41:05.477473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:05.477450    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-h6x8z_faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48/graceful-termination/0.log"
Jul 19 14:41:05.670880 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:05.670852    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-rzgsz_0805c402-8f62-4c36-9c47-9c66d82625da/kube-storage-version-migrator-operator/0.log"
Jul 19 14:41:06.106629 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:06.106594    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:06.321111 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:06.321072    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:06.635623 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:06.635563    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:06.736646 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:06.736616    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:06.904495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:06.904403    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:07.310906 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:07.310875    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:07.394851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:07.394816    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:07.946100 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:07.946056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:07.946100 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:07.946100    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:07.946528 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:07.946182    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:41:07.946528 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:07.946231    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:07.946528 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:07.946251    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:15.946235099 +0000 UTC m=+278.372712090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : configmap "kube-rbac-proxy" not found
Jul 19 14:41:07.946528 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:07.946272    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls podName:6530713d-747e-4cbe-a95b-f897147db574 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:15.946261032 +0000 UTC m=+278.372738023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tmgwr" (UID: "6530713d-747e-4cbe-a95b-f897147db574") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:41:08.110016 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.109984    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:08.318870 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.318838    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:08.418213 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.418179    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:08.708946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.708868    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:08.771558 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.771526    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:08.910960 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:08.910926    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:09.516513 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:09.516483    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:09.902223 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:09.902142    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:10.070512 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:10.070481    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:10.307159 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:10.307127    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:10.507094 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:10.507062    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:10.900752 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:10.900715    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:10.955841 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:10.955812    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:11.575131 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.575079    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:41:11.577549 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.577523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49005db1-e3a4-446e-b0e8-2e3994ae649b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7l949\" (UID: \"49005db1-e3a4-446e-b0e8-2e3994ae649b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:41:11.660136 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.660097    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"
Jul 19 14:41:11.788826 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.788763    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949"]
Jul 19 14:41:11.791734 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.791707    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:11.906478 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:11.906445    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:12.306440 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:12.306405    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:12.423957 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:12.423919    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949" event={"ID":"49005db1-e3a4-446e-b0e8-2e3994ae649b","Type":"ContainerStarted","Data":"b46557730a581d8e535d43e3410078a2fdc4b52ac84f61ae4782f5bd948249ea"}
Jul 19 14:41:12.550723 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:12.550693    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:12.707438 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:12.707344    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:13.040956 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:13.040922    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:13.703002 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:13.702965    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:13.903701 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:13.903676    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:14.173869 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.173836    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:14.274994 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.274953    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:14.345800 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.345742    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:14.432692 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.432658    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949" event={"ID":"49005db1-e3a4-446e-b0e8-2e3994ae649b","Type":"ContainerStarted","Data":"cce28bc36aaea7155c49551384ee765efcdae54ca438522a520cd4e6f3efac63"}
Jul 19 14:41:14.432886 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.432713    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949" event={"ID":"49005db1-e3a4-446e-b0e8-2e3994ae649b","Type":"ContainerStarted","Data":"bb7586a8e8bd38e4a649ea97a5b01a8e1a19ecd02aee1ee1f72c9bdd6e3d23ee"}
Jul 19 14:41:14.449487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.449432    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7l949" podStartSLOduration=17.410905826 podStartE2EDuration="19.449415267s" totalImagesPullingTime="2.038509441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:14.449176403 +0000 UTC m=+276.875653413" watchObservedRunningTime="2026-07-19 14:41:14.449415267 +0000 UTC m=+276.875892317"
Jul 19 14:41:14.511606 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.511570    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:14.903506 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:14.903475    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:15.302870 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:15.302840    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:15.550048 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:15.550017    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:16.013933 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.013891    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:16.013933 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.013934    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:16.014496 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.014470    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6530713d-747e-4cbe-a95b-f897147db574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:16.016420 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.016393    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6530713d-747e-4cbe-a95b-f897147db574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmgwr\" (UID: \"6530713d-747e-4cbe-a95b-f897147db574\") " pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:16.105622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.105582    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmgwr"
Jul 19 14:41:16.239155 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.239117    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmgwr"]
Jul 19 14:41:16.242165 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:16.242133    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6530713d_747e_4cbe_a95b_f897147db574.slice/crio-16193ff68c835c584431c1eab2c7806ef6e2c0c247df3893d0e283b8eacd7817 WatchSource:0}: Error finding container 16193ff68c835c584431c1eab2c7806ef6e2c0c247df3893d0e283b8eacd7817: Status 404 returned error can't find the container with id 16193ff68c835c584431c1eab2c7806ef6e2c0c247df3893d0e283b8eacd7817
Jul 19 14:41:16.440259 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:16.440171    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmgwr" event={"ID":"6530713d-747e-4cbe-a95b-f897147db574","Type":"ContainerStarted","Data":"16193ff68c835c584431c1eab2c7806ef6e2c0c247df3893d0e283b8eacd7817"}
Jul 19 14:41:18.498614 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:18.498560    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:18.599465 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:18.599427    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:18.699925 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:18.699882    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:18.801309 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:18.801208    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:18.973137 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:18.973098    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:19.028238 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.028201    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:19.081565 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:19.081479    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8lfqd" podUID="ca2399d4-f674-4415-aade-10149711f91c"
Jul 19 14:41:19.170340 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.170300    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:19.218385 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.218353    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:19.450611 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.450537    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:19.570432 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.570398    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:19.640449 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:19.640412    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:20.382130 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:20.382083    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:20.578931 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:20.578893    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:20.680182 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:20.680002    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:21.002346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:21.002309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:21.102348 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:21.102316    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:21.459257 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:21.459171    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmgwr" event={"ID":"6530713d-747e-4cbe-a95b-f897147db574","Type":"ContainerStarted","Data":"a550c6dda2a388ae83f1bf0024ddadf5600fb75bbfea62c34a71f56df701c03e"}
Jul 19 14:41:22.961292 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:22.961253    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:22.963742 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:22.963716    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca2399d4-f674-4415-aade-10149711f91c-metrics-tls\") pod \"dns-default-8lfqd\" (UID: \"ca2399d4-f674-4415-aade-10149711f91c\") " pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:23.053387 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:23.053355    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8vqq7\""
Jul 19 14:41:23.061505 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:23.061477    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:24.509331 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.509295    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"]
Jul 19 14:41:24.512894 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.512864    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:24.515963 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.515926    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 19 14:41:24.516133 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.515928    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5jhsh\""
Jul 19 14:41:24.523432 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.523402    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"]
Jul 19 14:41:24.575106 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.575066    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gkgvs\" (UID: \"1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:24.623147 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.623113    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:24.631653 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.631623    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"]
Jul 19 14:41:24.637474 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.636941    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.639632 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.639547    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rz27h\""
Jul 19 14:41:24.639997 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.639814    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 19 14:41:24.639997 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.639883    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 19 14:41:24.640170 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.640052    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 19 14:41:24.644076 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.644051    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 19 14:41:24.647168 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.647149    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"]
Jul 19 14:41:24.676182 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676156    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-tls\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676305 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676189    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-bound-sa-token\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676305 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676239    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-certificates\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676469 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676319    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-installation-pull-secrets\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676469 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676373    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-image-registry-private-configuration\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676469 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676407    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gkgvs\" (UID: \"1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:24.676469 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676452    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe4bafb-5264-4f7d-92d4-32098cc096fb-ca-trust-extracted\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676633 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676488    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxzvn\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-kube-api-access-dxzvn\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.676633 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.676529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-trusted-ca\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.678819 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.678765    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gkgvs\" (UID: \"1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:24.680288 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.680266    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8lfqd"]
Jul 19 14:41:24.683334 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:24.683286    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2399d4_f674_4415_aade_10149711f91c.slice/crio-1b67cabfeaad031b732a27728c87702645044e4dd1ca4bbda0caf5a86220ce99 WatchSource:0}: Error finding container 1b67cabfeaad031b732a27728c87702645044e4dd1ca4bbda0caf5a86220ce99: Status 404 returned error can't find the container with id 1b67cabfeaad031b732a27728c87702645044e4dd1ca4bbda0caf5a86220ce99
Jul 19 14:41:24.722528 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.722482    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:24.777346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777238    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxzvn\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-kube-api-access-dxzvn\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777297    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-trusted-ca\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777317    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-tls\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-bound-sa-token\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777694 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777504    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-certificates\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777694 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777577    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-installation-pull-secrets\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777694 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777634    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-image-registry-private-configuration\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.777871 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.777822    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe4bafb-5264-4f7d-92d4-32098cc096fb-ca-trust-extracted\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.778303 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.778246    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe4bafb-5264-4f7d-92d4-32098cc096fb-ca-trust-extracted\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.778458 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.778436    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-trusted-ca\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.778533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.778478    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-certificates\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.779946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.779925    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-registry-tls\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.780109 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.780084    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-installation-pull-secrets\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.780160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.780131    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe4bafb-5264-4f7d-92d4-32098cc096fb-image-registry-private-configuration\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.786804 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.786756    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-bound-sa-token\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.786900 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.786829    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxzvn\" (UniqueName: \"kubernetes.io/projected/dbe4bafb-5264-4f7d-92d4-32098cc096fb-kube-api-access-dxzvn\") pod \"image-registry-5cfdd5c6b5-jbmzd\" (UID: \"dbe4bafb-5264-4f7d-92d4-32098cc096fb\") " pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:24.825509 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.825476    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:24.951887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.951854    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"]
Jul 19 14:41:24.954818 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:24.954777    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f6d96fd_2bdf_404a_a1ea_69df1f9c80b1.slice/crio-d523eab118daee07706eac3df44bda7c5da2c7986e8ab1311c701cbf9a51696c WatchSource:0}: Error finding container d523eab118daee07706eac3df44bda7c5da2c7986e8ab1311c701cbf9a51696c: Status 404 returned error can't find the container with id d523eab118daee07706eac3df44bda7c5da2c7986e8ab1311c701cbf9a51696c
Jul 19 14:41:24.959311 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:24.959290    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:25.090192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.090158    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"]
Jul 19 14:41:25.093951 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:25.093900    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbe4bafb_5264_4f7d_92d4_32098cc096fb.slice/crio-5094aed687e5697e00832b2802342a94c5e2a7a3e29fb3a0e8219134286706da WatchSource:0}: Error finding container 5094aed687e5697e00832b2802342a94c5e2a7a3e29fb3a0e8219134286706da: Status 404 returned error can't find the container with id 5094aed687e5697e00832b2802342a94c5e2a7a3e29fb3a0e8219134286706da
Jul 19 14:41:25.476216 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.476061    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd" event={"ID":"dbe4bafb-5264-4f7d-92d4-32098cc096fb","Type":"ContainerStarted","Data":"3e3eced3955d1bbff76b0864113e0d8da1f3fc2eef3157c3865fc77bdfcd1f5a"}
Jul 19 14:41:25.476216 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.476130    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd" event={"ID":"dbe4bafb-5264-4f7d-92d4-32098cc096fb","Type":"ContainerStarted","Data":"5094aed687e5697e00832b2802342a94c5e2a7a3e29fb3a0e8219134286706da"}
Jul 19 14:41:25.476216 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.476174    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:25.477757 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.477701    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs" event={"ID":"1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1","Type":"ContainerStarted","Data":"d523eab118daee07706eac3df44bda7c5da2c7986e8ab1311c701cbf9a51696c"}
Jul 19 14:41:25.479907 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.479875    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lfqd" event={"ID":"ca2399d4-f674-4415-aade-10149711f91c","Type":"ContainerStarted","Data":"1b67cabfeaad031b732a27728c87702645044e4dd1ca4bbda0caf5a86220ce99"}
Jul 19 14:41:25.482149 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.482116    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmgwr" event={"ID":"6530713d-747e-4cbe-a95b-f897147db574","Type":"ContainerStarted","Data":"fac21c65de9a3e2560d3ac52dc9f0acd42b9454c3156e6a3de3e5f42ada43f98"}
Jul 19 14:41:25.496303 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:25.496208    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd" podStartSLOduration=1.496186491 podStartE2EDuration="1.496186491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:25.494679745 +0000 UTC m=+287.921156759" watchObservedRunningTime="2026-07-19 14:41:25.496186491 +0000 UTC m=+287.922663505"
Jul 19 14:41:28.495800 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:28.495700    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs" event={"ID":"1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1","Type":"ContainerStarted","Data":"dade741b02426ab2f38723fc7b1ad9e23f4535a5e49b7ea298f3a043376f67e3"}
Jul 19 14:41:28.496234 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:28.496057    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:28.500987 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:28.500954    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs"
Jul 19 14:41:28.511215 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:28.511162    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gkgvs" podStartSLOduration=1.09205752 podStartE2EDuration="4.511142469s" totalImagesPullingTime="3.419084949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:28.511106532 +0000 UTC m=+290.937583547" watchObservedRunningTime="2026-07-19 14:41:28.511142469 +0000 UTC m=+290.937619483"
Jul 19 14:41:29.309636 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.309605    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:29.401093 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.401063    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:29.501277 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.501238    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lfqd" event={"ID":"ca2399d4-f674-4415-aade-10149711f91c","Type":"ContainerStarted","Data":"67606860c67457ad69231c30f0217a9b2f2e630f797b8b0f230361bb8f74d698"}
Jul 19 14:41:29.501723 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.501296    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lfqd" event={"ID":"ca2399d4-f674-4415-aade-10149711f91c","Type":"ContainerStarted","Data":"cabd0deec9426e991bacfbf71c0357ab9b36276ec647a8fca1262cafe1798646"}
Jul 19 14:41:29.501723 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.501322    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:29.503365 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.503334    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmgwr" event={"ID":"6530713d-747e-4cbe-a95b-f897147db574","Type":"ContainerStarted","Data":"b0a9e939b70f26504cd16b551165d27c28a8778a9d15c935f8b8e994be611b83"}
Jul 19 14:41:29.517935 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.517882    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8lfqd" podStartSLOduration=253.827634148 podStartE2EDuration="4m17.517863933s" totalImagesPullingTime="3.690229785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:29.516991908 +0000 UTC m=+291.943468921" watchObservedRunningTime="2026-07-19 14:41:29.517863933 +0000 UTC m=+291.944340940"
Jul 19 14:41:29.533110 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:29.533054    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmgwr" podStartSLOduration=17.465085125 podStartE2EDuration="29.533034542s" totalImagesPullingTime="12.067949417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:41:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:29.532487927 +0000 UTC m=+291.958964943" watchObservedRunningTime="2026-07-19 14:41:29.533034542 +0000 UTC m=+291.959511556"
Jul 19 14:41:30.675264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:30.675227    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:31.418203 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:31.418152    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vftg8" podUID="223f7083-66c8-4efb-9c2e-f19a4b4724be"
Jul 19 14:41:32.389946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:32.389910    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:32.613772 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:32.613738    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:32.714734 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:32.714639    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.171325 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.171282    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.228319 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.228285    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.419573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.419530    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.634959 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.634920    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.671584 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.671555    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-8zswf"]
Jul 19 14:41:34.676000 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.675976    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-68zx8"]
Jul 19 14:41:34.676128 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.676094    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.678828 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.678809    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 19 14:41:34.679046 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.679030    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 19 14:41:34.679275 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.679258    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.679409 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.679385    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 19 14:41:34.679839 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.679825    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 19 14:41:34.680641 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.680623    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 19 14:41:34.681582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681563    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 19 14:41:34.681582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681575    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zt9dh\""
Jul 19 14:41:34.681746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681582    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 19 14:41:34.681746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681606    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 19 14:41:34.681746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681680    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-trcrn\""
Jul 19 14:41:34.681897 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.681814    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 19 14:41:34.690300 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.690278    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-8zswf"]
Jul 19 14:41:34.727098 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.727069    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.757413 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757378    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkn7c\" (UniqueName: \"kubernetes.io/projected/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-kube-api-access-rkn7c\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.757413 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757416    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757493    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-root\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757591    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757618    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.757686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757665    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae2f832f-1cac-4fe5-9697-35a754873776-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757690    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-sys\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757715    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-tls\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757815    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-textfile\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757840    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-wtmp\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757873    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-metrics-client-ca\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757910    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.758055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.757937    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdd65\" (UniqueName: \"kubernetes.io/projected/ae2f832f-1cac-4fe5-9697-35a754873776-kube-api-access-gdd65\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.796460 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.796426    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.850014 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.849978    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.858691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858666    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.858851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.858851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858714    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.858851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858732    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-root\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.858851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.858851 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858843    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-root\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859115 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858892    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859115 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae2f832f-1cac-4fe5-9697-35a754873776-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859115 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.858974    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-sys\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859115 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859001    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-tls\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859115 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:34.859063    2585 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 19 14:41:34.859375 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:34.859133    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls podName:ae2f832f-1cac-4fe5-9697-35a754873776 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:35.35911113 +0000 UTC m=+297.785588125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-8zswf" (UID: "ae2f832f-1cac-4fe5-9697-35a754873776") : secret "kube-state-metrics-tls" not found
Jul 19 14:41:34.859375 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859317    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ae2f832f-1cac-4fe5-9697-35a754873776-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859405    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-sys\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-textfile\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859448    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-wtmp\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859452    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-accelerators-collector-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859478    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-metrics-client-ca\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859513    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859544    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdd65\" (UniqueName: \"kubernetes.io/projected/ae2f832f-1cac-4fe5-9697-35a754873776-kube-api-access-gdd65\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.859737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859572    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkn7c\" (UniqueName: \"kubernetes.io/projected/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-kube-api-access-rkn7c\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.859737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859596    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-wtmp\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.860026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.859948    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-metrics-client-ca\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.860219 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.860190    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.860323 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.860304    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-textfile\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.861462 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.861442    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.861771 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.861755    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-tls\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.861897 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.861880    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.869702 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.869683    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkn7c\" (UniqueName: \"kubernetes.io/projected/f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38-kube-api-access-rkn7c\") pod \"node-exporter-68zx8\" (UID: \"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38\") " pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:34.870150 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.870130    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdd65\" (UniqueName: \"kubernetes.io/projected/ae2f832f-1cac-4fe5-9697-35a754873776-kube-api-access-gdd65\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:34.931515 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.931433    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:34.992478 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:34.992434    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68zx8"
Jul 19 14:41:35.004100 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:35.004070    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73c8d9a_a5f7_481a_950f_ec2d7e2a0c38.slice/crio-7b10e161bc5020971ef5308b6b2aa6bc2bddadb01a152a8def5cf4049be9d07d WatchSource:0}: Error finding container 7b10e161bc5020971ef5308b6b2aa6bc2bddadb01a152a8def5cf4049be9d07d: Status 404 returned error can't find the container with id 7b10e161bc5020971ef5308b6b2aa6bc2bddadb01a152a8def5cf4049be9d07d
Jul 19 14:41:35.040277 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.040243    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.131887 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.131857    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.239688 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.239654    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.363373 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.363330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:35.365768 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.365748    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae2f832f-1cac-4fe5-9697-35a754873776-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-8zswf\" (UID: \"ae2f832f-1cac-4fe5-9697-35a754873776\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:35.527671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.527592    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68zx8" event={"ID":"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38","Type":"ContainerStarted","Data":"7b10e161bc5020971ef5308b6b2aa6bc2bddadb01a152a8def5cf4049be9d07d"}
Jul 19 14:41:35.587145 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.587101    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf"
Jul 19 14:41:35.658499 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.658465    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.741160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.741043    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-8zswf"]
Jul 19 14:41:35.743403 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:35.743364    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2f832f_1cac_4fe5_9697_35a754873776.slice/crio-4720d680d427593f70652e3dc900332a18b5d30f6fbc9db5a9ad09fb2bf47b44 WatchSource:0}: Error finding container 4720d680d427593f70652e3dc900332a18b5d30f6fbc9db5a9ad09fb2bf47b44: Status 404 returned error can't find the container with id 4720d680d427593f70652e3dc900332a18b5d30f6fbc9db5a9ad09fb2bf47b44
Jul 19 14:41:35.761819 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.761757    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:35.858051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:35.857933    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:36.059626 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:36.059586    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:36.125606 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:36.125509    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:36.533200 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:36.533158    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf" event={"ID":"ae2f832f-1cac-4fe5-9697-35a754873776","Type":"ContainerStarted","Data":"4720d680d427593f70652e3dc900332a18b5d30f6fbc9db5a9ad09fb2bf47b44"}
Jul 19 14:41:36.535372 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:36.535344    2585 generic.go:356] "Generic (PLEG): container finished" podID="f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38" containerID="1cf2001acd3027a3ecd91896a710348fde41473c8c7b5ff462072192c9f3ee76" exitCode=0
Jul 19 14:41:36.535497 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:36.535438    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68zx8" event={"ID":"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38","Type":"ContainerDied","Data":"1cf2001acd3027a3ecd91896a710348fde41473c8c7b5ff462072192c9f3ee76"}
Jul 19 14:41:37.248152 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.248128    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:37.348434 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.348400    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:37.541971 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.541917    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68zx8" event={"ID":"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38","Type":"ContainerStarted","Data":"d63274af550c78704d19f316d84bd32a2935a1295312945a7af6c67c1984d3ea"}
Jul 19 14:41:37.542473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.542005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68zx8" event={"ID":"f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38","Type":"ContainerStarted","Data":"64190c35dec943e3b203d42978f3ed5e01df359ddf6354df8f73f5c2e61c963d"}
Jul 19 14:41:37.544593 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.544550    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf" event={"ID":"ae2f832f-1cac-4fe5-9697-35a754873776","Type":"ContainerStarted","Data":"7d2d3334fd5c7803431dcc01886f1bb050e6cd45d8b6b42f7d5a376b794d62a6"}
Jul 19 14:41:37.544718 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.544610    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf" event={"ID":"ae2f832f-1cac-4fe5-9697-35a754873776","Type":"ContainerStarted","Data":"a41e7d3da23165a7473aec88db866a1e4980825f65f2cd79c983dede82eb536f"}
Jul 19 14:41:37.544718 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.544638    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf" event={"ID":"ae2f832f-1cac-4fe5-9697-35a754873776","Type":"ContainerStarted","Data":"7b65a8919feb449fd51bf01e13ae0334191dcff31fad7fe0e53937ac5cbe7f84"}
Jul 19 14:41:37.565721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.565442    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-68zx8" podStartSLOduration=2.297939324 podStartE2EDuration="3.565418125s" totalImagesPullingTime="1.203389521s" totalInitContainerRuntime="64.08928ms" isStatefulPod=true podCreationTimestamp="2026-07-19 14:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:37.564062398 +0000 UTC m=+299.990539410" watchObservedRunningTime="2026-07-19 14:41:37.565418125 +0000 UTC m=+299.991895141"
Jul 19 14:41:37.582147 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.582069    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-8zswf" podStartSLOduration=2.147468357 podStartE2EDuration="3.582052785s" totalImagesPullingTime="1.434584428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:37.58165906 +0000 UTC m=+300.008136097" watchObservedRunningTime="2026-07-19 14:41:37.582052785 +0000 UTC m=+300.008529798"
Jul 19 14:41:37.583599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.583566    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:37.681077 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.681047    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:37.767826 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.767773    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:37.949969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:37.949881    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:38.065430 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:38.065402    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:41:38.065644 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:38.065628    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:41:38.072880 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:38.072852    2585 kubelet.go:1648] "Image garbage collection succeeded"
Jul 19 14:41:39.443461 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.443417    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:39.446645 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.446618    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"]
Jul 19 14:41:39.449675 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.449657    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.451964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.451944    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 19 14:41:39.452029 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.451949    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s7zkd\""
Jul 19 14:41:39.456996 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.456967    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"]
Jul 19 14:41:39.498792 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.498754    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8slt\" (UniqueName: \"kubernetes.io/projected/7ca1b754-a62e-4629-a90f-20ccdfeecb43-kube-api-access-h8slt\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.498977 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.498849    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.509692 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.509658    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8lfqd"
Jul 19 14:41:39.532341 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.532307    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:39.599992 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.599947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.600198 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:39.600070    2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 19 14:41:39.600198 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.600088    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8slt\" (UniqueName: \"kubernetes.io/projected/7ca1b754-a62e-4629-a90f-20ccdfeecb43-kube-api-access-h8slt\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.600198 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:41:39.600139    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert podName:7ca1b754-a62e-4629-a90f-20ccdfeecb43 nodeName:}" failed. No retries permitted until 2026-07-19 14:41:40.100119811 +0000 UTC m=+302.526596805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-wnckx" (UID: "7ca1b754-a62e-4629-a90f-20ccdfeecb43") : secret "monitoring-plugin-cert" not found
Jul 19 14:41:39.610286 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.610246    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8slt\" (UniqueName: \"kubernetes.io/projected/7ca1b754-a62e-4629-a90f-20ccdfeecb43-kube-api-access-h8slt\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:39.623055 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.623023    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:39.674377 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:39.674343    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:40.104828 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.104766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:40.107201 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.107176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7ca1b754-a62e-4629-a90f-20ccdfeecb43-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnckx\" (UID: \"7ca1b754-a62e-4629-a90f-20ccdfeecb43\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:40.121975 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.121950    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:40.359121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.359005    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:40.496370 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.496333    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"]
Jul 19 14:41:40.499438 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:40.499405    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca1b754_a62e_4629_a90f_20ccdfeecb43.slice/crio-3a2cfb01c8c8ed09329920bfc9561ce45903f870ef1f373d680ae4d32059320a WatchSource:0}: Error finding container 3a2cfb01c8c8ed09329920bfc9561ce45903f870ef1f373d680ae4d32059320a: Status 404 returned error can't find the container with id 3a2cfb01c8c8ed09329920bfc9561ce45903f870ef1f373d680ae4d32059320a
Jul 19 14:41:40.501840 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.501821    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:41:40.557844 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.557770    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx" event={"ID":"7ca1b754-a62e-4629-a90f-20ccdfeecb43","Type":"ContainerStarted","Data":"3a2cfb01c8c8ed09329920bfc9561ce45903f870ef1f373d680ae4d32059320a"}
Jul 19 14:41:40.653231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.653150    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:41:40.657466 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.657444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.661811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.661771    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 19 14:41:40.661953 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.661771    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 19 14:41:40.661953 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.661831    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 19 14:41:40.663011 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.662978    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 19 14:41:40.663165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.663060    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 19 14:41:40.663165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.663101    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 19 14:41:40.663165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.662978    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lzfm7\""
Jul 19 14:41:40.663331 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.663292    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 19 14:41:40.671746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.671661    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 19 14:41:40.674920 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.674899    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:41:40.709864 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.709827    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710043 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.709881    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710043 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.709971    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710043 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.709992    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkv6d\" (UniqueName: \"kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710043 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.710036    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710199 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.710057    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.710199 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.710160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.810895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.810852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.810895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.810896    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.810911    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkv6d\" (UniqueName: \"kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.810944    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.810961    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.811075    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811160 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.811110    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811771 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.811739    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811922 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.811903    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.811981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.811917    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.812037 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.812016    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.813708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.813682    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.813831 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.813682    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.823418 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.823386    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkv6d\" (UniqueName: \"kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d\") pod \"console-b49b79558-q626s\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") " pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.956885 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.956807    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:41:40.961979 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.961955    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:40.963908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.963882    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:40.966173 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966153    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 19 14:41:40.966407 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966388    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 19 14:41:40.966491 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966418    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 19 14:41:40.966662 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966642    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 19 14:41:40.966811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966645    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 19 14:41:40.966811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966645    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 19 14:41:40.966811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966685    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 19 14:41:40.966811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966763    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 19 14:41:40.967038 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.966903    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7h33q5jn792ms\""
Jul 19 14:41:40.967109 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.967097    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 19 14:41:40.967154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.967125    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 19 14:41:40.967192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.967126    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 19 14:41:40.967394 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.967380    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 19 14:41:40.967700 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.967683    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vn4x6\""
Jul 19 14:41:40.969593 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.969571    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 19 14:41:40.972735 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.972714    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:40.972905 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.972888    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 19 14:41:40.974549 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.974531    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 19 14:41:40.984114 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:40.984091    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:41:41.012481 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012426    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.012674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.012674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.012674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012553    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.012674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012631    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.012674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012655    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012681    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012726    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012746    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012760    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012803    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012827    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012871    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.012956    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013039    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013089    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013110    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knhmg\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013140    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.013686 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.013179    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113618    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113656    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113677    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113703    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113722    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knhmg\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113769    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113818    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113843    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113870    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113899    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.113955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.113981    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114002    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114046    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114095    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114133    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114157    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114431 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114184    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114826 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114524    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.114826 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.114691    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.115484 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.115455    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.116144 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.116113    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.117230 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.117202    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.118270 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.117723    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.118270 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.118193    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.118658 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.118632    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.119268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.119186    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.119268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.119213    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.119268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.119263    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120424 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.119624    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120424 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.119651    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120424 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.120276    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.120511    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120651 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.120523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120757 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.120701    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.120917 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.120891    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:41:41.121698 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.121671    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.121867 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.121702    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.122357 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.122340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.122824 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:41.122798    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda343f716_6e1d_4970_b617_824005544b13.slice/crio-16b16fecab9c3f385c1e35edc0c96cd749096625ba842000dbd60190012556cf WatchSource:0}: Error finding container 16b16fecab9c3f385c1e35edc0c96cd749096625ba842000dbd60190012556cf: Status 404 returned error can't find the container with id 16b16fecab9c3f385c1e35edc0c96cd749096625ba842000dbd60190012556cf
Jul 19 14:41:41.122929 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.122917    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knhmg\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg\") pod \"prometheus-k8s-0\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.181602 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.181569    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.272969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.272932    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:41.281994 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.281959    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.429844 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.429806    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:41:41.433614 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:41:41.433563    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d5e9e5_9a18_4914_8206_945d77f70bca.slice/crio-3431d18f3ca1f6accd882fbaf213d95f83571c60b8d66042f0fe6c93ac52b3a2 WatchSource:0}: Error finding container 3431d18f3ca1f6accd882fbaf213d95f83571c60b8d66042f0fe6c93ac52b3a2: Status 404 returned error can't find the container with id 3431d18f3ca1f6accd882fbaf213d95f83571c60b8d66042f0fe6c93ac52b3a2
Jul 19 14:41:41.564482 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.564368    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49b79558-q626s" event={"ID":"a343f716-6e1d-4970-b617-824005544b13","Type":"ContainerStarted","Data":"16b16fecab9c3f385c1e35edc0c96cd749096625ba842000dbd60190012556cf"}
Jul 19 14:41:41.566308 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.566263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"3431d18f3ca1f6accd882fbaf213d95f83571c60b8d66042f0fe6c93ac52b3a2"}
Jul 19 14:41:41.621528 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.621493    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:41.723950 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:41.723888    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:42.156403 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.156317    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:42.252753 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.252712    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:42.574045 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.574005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx" event={"ID":"7ca1b754-a62e-4629-a90f-20ccdfeecb43","Type":"ContainerStarted","Data":"0237f8f726bc6ae25dcc977cec03261e9d4aaf5031b93e859598719ac9dcffa7"}
Jul 19 14:41:42.574471 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.574227    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:42.580241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.580189    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx"
Jul 19 14:41:42.610220 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.610156    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnckx" podStartSLOduration=1.993801533 podStartE2EDuration="3.610135244s" totalImagesPullingTime="1.616333711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:42.590152508 +0000 UTC m=+305.016629521" watchObservedRunningTime="2026-07-19 14:41:42.610135244 +0000 UTC m=+305.036612260"
Jul 19 14:41:42.641393 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.641361    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:42.740624 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:42.740591    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:44.586041 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:44.585991    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49b79558-q626s" event={"ID":"a343f716-6e1d-4970-b617-824005544b13","Type":"ContainerStarted","Data":"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"}
Jul 19 14:41:44.587495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:44.587471    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d" exitCode=0
Jul 19 14:41:44.587630 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:44.587523    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"}
Jul 19 14:41:44.601614 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:44.601555    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b49b79558-q626s" podStartSLOduration=1.800529281 podStartE2EDuration="4.601535487s" totalImagesPullingTime="2.801006206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:44.601263928 +0000 UTC m=+307.027740944" watchObservedRunningTime="2026-07-19 14:41:44.601535487 +0000 UTC m=+307.028012499"
Jul 19 14:41:45.416819 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:45.416768    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:41:46.357516 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.357477    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:46.451304 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.451261    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:46.491341 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.491312    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cfdd5c6b5-jbmzd"
Jul 19 14:41:46.619400 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.619309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:46.727314 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.727275    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:46.782596 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:46.782553    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:48.607477 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:48.607438    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"}
Jul 19 14:41:48.607893 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:48.607521    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"}
Jul 19 14:41:48.972871 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:48.972719    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:49.045317 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:49.045272    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:49.614172 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:49.614139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"}
Jul 19 14:41:49.614507 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:49.614264    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"}
Jul 19 14:41:50.621338 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:50.621303    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"}
Jul 19 14:41:50.621744 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:50.621369    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerStarted","Data":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"}
Jul 19 14:41:50.648276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:50.648213    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.551983044 podStartE2EDuration="10.648191857s" totalImagesPullingTime="6.076633946s" totalInitContainerRuntime="19.574867ms" isStatefulPod=false podCreationTimestamp="2026-07-19 14:41:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:41:50.646671969 +0000 UTC m=+313.073148982" watchObservedRunningTime="2026-07-19 14:41:50.648191857 +0000 UTC m=+313.074668873"
Jul 19 14:41:50.919632 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:50.919539    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:41:50.973301 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:50.973262    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:41:51.111479 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:51.111443    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:51.211861 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:51.211745    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:51.273217 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:51.273183    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:41:51.713649 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:51.713592    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:53.054489 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:53.054442    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:41:53.105371 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:41:53.105339    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:02.191418 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:02.191381    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:02.292757 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:02.292722    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:02.348136 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:02.348099    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:03.979315 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:03.979281    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:04.027838 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:04.027803    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:08.691713 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:08.691676    2585 generic.go:356] "Generic (PLEG): container finished" podID="0805c402-8f62-4c36-9c47-9c66d82625da" containerID="8322fef1f30560839d07e741de0d3a793888b32249387509c726c985fa09857e" exitCode=0
Jul 19 14:42:08.692121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:08.691726    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz" event={"ID":"0805c402-8f62-4c36-9c47-9c66d82625da","Type":"ContainerDied","Data":"8322fef1f30560839d07e741de0d3a793888b32249387509c726c985fa09857e"}
Jul 19 14:42:08.692121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:08.692046    2585 scope.go:117] "RemoveContainer" containerID="8322fef1f30560839d07e741de0d3a793888b32249387509c726c985fa09857e"
Jul 19 14:42:09.697545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:09.697511    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rzgsz" event={"ID":"0805c402-8f62-4c36-9c47-9c66d82625da","Type":"ContainerStarted","Data":"e81f50525598b3c21eac1ebf1e6f3bb4183ec629516d0979bead8ea1be7ed95a"}
Jul 19 14:42:15.952324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:15.952278    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b49b79558-q626s" podUID="a343f716-6e1d-4970-b617-824005544b13" containerName="console" containerID="cri-o://6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53" gracePeriod=15
Jul 19 14:42:16.209994 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.209929    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:42:16.232618 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232586    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.232823 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232646    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkv6d\" (UniqueName: \"kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.232902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232821    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.232902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232868    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.232902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232890    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.233054 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232954    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.233054 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.232983    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca\") pod \"a343f716-6e1d-4970-b617-824005544b13\" (UID: \"a343f716-6e1d-4970-b617-824005544b13\") "
Jul 19 14:42:16.233213 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.233179    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.233286    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.233608    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca" (OuterVolumeSpecName: "service-ca") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.233902    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config" (OuterVolumeSpecName: "console-config") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.233990    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-trusted-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.234012    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-oauth-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.234051 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.234027    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.235423 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.235390    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:16.235531 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.235427    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:16.235531 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.235494    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d" (OuterVolumeSpecName: "kube-api-access-bkv6d") pod "a343f716-6e1d-4970-b617-824005544b13" (UID: "a343f716-6e1d-4970-b617-824005544b13"). InnerVolumeSpecName "kube-api-access-bkv6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:42:16.335398 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.335355    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkv6d\" (UniqueName: \"kubernetes.io/projected/a343f716-6e1d-4970-b617-824005544b13-kube-api-access-bkv6d\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.335398 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.335389    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-oauth-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.335398 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.335399    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a343f716-6e1d-4970-b617-824005544b13-console-serving-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.335398 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.335410    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a343f716-6e1d-4970-b617-824005544b13-console-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:16.726751 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.726709    2585 generic.go:356] "Generic (PLEG): container finished" podID="a343f716-6e1d-4970-b617-824005544b13" containerID="6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53" exitCode=0
Jul 19 14:42:16.726978 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.726768    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b49b79558-q626s"
Jul 19 14:42:16.726978 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.726804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49b79558-q626s" event={"ID":"a343f716-6e1d-4970-b617-824005544b13","Type":"ContainerDied","Data":"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"}
Jul 19 14:42:16.726978 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.726884    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b49b79558-q626s" event={"ID":"a343f716-6e1d-4970-b617-824005544b13","Type":"ContainerDied","Data":"16b16fecab9c3f385c1e35edc0c96cd749096625ba842000dbd60190012556cf"}
Jul 19 14:42:16.726978 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.726915    2585 scope.go:117] "RemoveContainer" containerID="6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"
Jul 19 14:42:16.735211 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.735188    2585 scope.go:117] "RemoveContainer" containerID="6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"
Jul 19 14:42:16.735502 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:16.735478    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53\": container with ID starting with 6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53 not found: ID does not exist" containerID="6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"
Jul 19 14:42:16.735596 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.735510    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53"} err="failed to get container status \"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53\": rpc error: code = NotFound desc = could not find container \"6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53\": container with ID starting with 6986bf486ed89b9b943a5abcd818945a367a9911a4c2aaa7967b8fefd8d9bc53 not found: ID does not exist"
Jul 19 14:42:16.742489 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.742464    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:42:16.746081 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:16.746057    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b49b79558-q626s"]
Jul 19 14:42:18.421526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:18.421491    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a343f716-6e1d-4970-b617-824005544b13" path="/var/lib/kubelet/pods/a343f716-6e1d-4970-b617-824005544b13/volumes"
Jul 19 14:42:41.273746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:41.273708    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:41.293217 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:41.293189    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:41.845734 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:41.845706    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:44.390400 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:44.390370    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:44.447189 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:44.447154    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:47.329625 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:47.329584    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:47.391053 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:47.391009    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:47.444232 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:47.444176    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:50.779564 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:50.779525    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:50.829432 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:50.829398    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:51.046114 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.046019    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:42:51.048353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.048323    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223f7083-66c8-4efb-9c2e-f19a4b4724be-metrics-certs\") pod \"network-metrics-daemon-vftg8\" (UID: \"223f7083-66c8-4efb-9c2e-f19a4b4724be\") " pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:42:51.119507 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.119470    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f5r6f\""
Jul 19 14:42:51.127375 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.127347    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vftg8"
Jul 19 14:42:51.261850 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.261814    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vftg8"]
Jul 19 14:42:51.264766 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:42:51.264720    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223f7083_66c8_4efb_9c2e_f19a4b4724be.slice/crio-054387c5cfd684a7dfc21f3579563dbf5e0bd8de023fc41447c0c66a5f70d72b WatchSource:0}: Error finding container 054387c5cfd684a7dfc21f3579563dbf5e0bd8de023fc41447c0c66a5f70d72b: Status 404 returned error can't find the container with id 054387c5cfd684a7dfc21f3579563dbf5e0bd8de023fc41447c0c66a5f70d72b
Jul 19 14:42:51.868459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:51.868419    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vftg8" event={"ID":"223f7083-66c8-4efb-9c2e-f19a4b4724be","Type":"ContainerStarted","Data":"054387c5cfd684a7dfc21f3579563dbf5e0bd8de023fc41447c0c66a5f70d72b"}
Jul 19 14:42:52.873704 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:52.873669    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vftg8" event={"ID":"223f7083-66c8-4efb-9c2e-f19a4b4724be","Type":"ContainerStarted","Data":"fb2428ed2ab20a2b9206a1668c1ae5b096e8cd9889b1b88a706596a4c8495f7a"}
Jul 19 14:42:52.874116 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:52.873726    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vftg8" event={"ID":"223f7083-66c8-4efb-9c2e-f19a4b4724be","Type":"ContainerStarted","Data":"60d64a19d08f0dd5962db881519787836d706ee92eb42b1652e244123b45cea5"}
Jul 19 14:42:52.889470 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:52.889418    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vftg8" podStartSLOduration=374.056521919 podStartE2EDuration="6m14.889400303s" totalImagesPullingTime="832.878384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:42:52.887771432 +0000 UTC m=+375.314248444" watchObservedRunningTime="2026-07-19 14:42:52.889400303 +0000 UTC m=+375.315877315"
Jul 19 14:42:52.968017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:52.967974    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:53.020685 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:53.020632    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:42:59.224012 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.223969    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:42:59.224483 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224360    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="prometheus" containerID="cri-o://f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615" gracePeriod=600
Jul 19 14:42:59.224483 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224395    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="config-reloader" containerID="cri-o://92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835" gracePeriod=600
Jul 19 14:42:59.224483 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224395    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy" containerID="cri-o://6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495" gracePeriod=600
Jul 19 14:42:59.224483 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224433    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-web" containerID="cri-o://10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364" gracePeriod=600
Jul 19 14:42:59.224683 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224403    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="thanos-sidecar" containerID="cri-o://963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5" gracePeriod=600
Jul 19 14:42:59.224683 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.224409    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-thanos" containerID="cri-o://86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae" gracePeriod=600
Jul 19 14:42:59.477713 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.477686    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:59.515198 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515168    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.515392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515207    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.515392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515256    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.515392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515294    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.515392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515338    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515702    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515719    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515772    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515837    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515864    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515889    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515926    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515955    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.515982    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516019    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516043    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516068    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knhmg\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516098    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516109    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516124    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.516527 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516498    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.516892 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.516691    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:42:59.517473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.517126    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.517473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.517463    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.519255 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.518939    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.519382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519263    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.519382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519277    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.519382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519305    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"77d5e9e5-9a18-4914-8206-945d77f70bca\" (UID: \"77d5e9e5-9a18-4914-8206-945d77f70bca\") "
Jul 19 14:42:59.519382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519328    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519716    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519740    2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-metrics-client-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519754    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-db\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519750    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519768    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519799    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519816    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519831    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519845    2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.519859    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-metrics-client-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.520394    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg" (OuterVolumeSpecName: "kube-api-access-knhmg") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "kube-api-access-knhmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.521076    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out" (OuterVolumeSpecName: "config-out") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.521326    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config" (OuterVolumeSpecName: "config") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.521365    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.522241 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.521425    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.523065 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.521654    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.523065 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.522294    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:42:59.523065 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.522633    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.523065 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.522962    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:42:59.533749 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.533715    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config" (OuterVolumeSpecName: "web-config") pod "77d5e9e5-9a18-4914-8206-945d77f70bca" (UID: "77d5e9e5-9a18-4914-8206-945d77f70bca"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:42:59.620349 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620302    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620349 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620339    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/77d5e9e5-9a18-4914-8206-945d77f70bca-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620349 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620349    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-web-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620349 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620358    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620370    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77d5e9e5-9a18-4914-8206-945d77f70bca-config-out\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620380    2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-config\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620388    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-tls-assets\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620396    2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-grpc-tls\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620405    2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-kube-rbac-proxy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620413    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/77d5e9e5-9a18-4914-8206-945d77f70bca-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.620622 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.620422    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knhmg\" (UniqueName: \"kubernetes.io/projected/77d5e9e5-9a18-4914-8206-945d77f70bca-kube-api-access-knhmg\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904366    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae" exitCode=0
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904396    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495" exitCode=0
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904406    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364" exitCode=0
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904415    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5" exitCode=0
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904411    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"}
Jul 19 14:42:59.904463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904446    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904489    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"}
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904424    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835" exitCode=0
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904516    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615" exitCode=0
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904529    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904518    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"}
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904609    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"}
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904650    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"}
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904695    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"}
Jul 19 14:42:59.904908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.904738    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"77d5e9e5-9a18-4914-8206-945d77f70bca","Type":"ContainerDied","Data":"3431d18f3ca1f6accd882fbaf213d95f83571c60b8d66042f0fe6c93ac52b3a2"}
Jul 19 14:42:59.913809 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.913767    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.921650 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.921629    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.929320 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.929283    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:42:59.930596 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.930550    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.932463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.932442    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:42:59.937877 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.937853    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.945060 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.945042    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.952916 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.952896    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.960195 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.960159    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.960937 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.960647    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.960937 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.960693    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.960937 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.960720    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.961153 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.961064    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.961153 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961091    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.961153 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961113    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.961415 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.961388    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.961415 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961417    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.961415 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961439    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.961753 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.961733    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.961825 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961758    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.961825 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.961774    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.962101 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.962081    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.962173 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962105    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.962173 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962130    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.962377 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.962358    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.962434 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962381    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.962434 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962394    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.962661 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:42:59.962643    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.962720 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962665    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.962720 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962678    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.962930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962908    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.962930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962929    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.963108 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.962941    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:42:59.963180 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963162    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.963221 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963180    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.963267 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963251    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="config-reloader"
Jul 19 14:42:59.963267 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963263    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="config-reloader"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963270    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963278    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963294    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="init-config-reloader"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963302    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="init-config-reloader"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963313    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-web"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963318    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-web"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963328    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a343f716-6e1d-4970-b617-824005544b13" containerName="console"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963333    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a343f716-6e1d-4970-b617-824005544b13" containerName="console"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963339    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="prometheus"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963344    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="prometheus"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963351    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="thanos-sidecar"
Jul 19 14:42:59.963362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963361    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="thanos-sidecar"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963369    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-thanos"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963378    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-thanos"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963387    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963402    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963423    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="thanos-sidecar"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963434    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-web"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963444    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="config-reloader"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963450    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="prometheus"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963454    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963460    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" containerName="kube-rbac-proxy-thanos"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963465    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a343f716-6e1d-4970-b617-824005544b13" containerName="console"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963583    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.963774 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963596    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.964205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963815    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.964205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.963830    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.964205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964039    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.964205 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964055    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.964344 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964234    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.964344 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964251    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.964471 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964456    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.964510 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964472    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.964677 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964657    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.964753 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964678    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.964909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964886    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.964909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.964909    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.965129 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965113    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.965170 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965129    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.965324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965308    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.965362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965326    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.965516 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965500    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.965553 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965517    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.965693 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965675    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.965737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965695    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.965923 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965905    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.965974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.965923    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.966146 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966117    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.966146 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966145    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.966391 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966336    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.966466 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966393    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.966667 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966641    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.966721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966666    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.966964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966940    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.966964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.966961    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.967214 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967196    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.967258 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967214    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.967411 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967394    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.967484 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967413    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.967643 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967626    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.967691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967644    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.967873 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967856    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.967948 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.967902    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.968140 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968120    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.968140 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968138    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.968357 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968338    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.968427 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968357    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.968565 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968549    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.968608 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968564    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.968837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968768    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.968837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968807    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.968837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.968773    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:42:59.969056 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969028    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.969118 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969056    2585 scope.go:117] "RemoveContainer" containerID="86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"
Jul 19 14:42:59.969395 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969365    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae"} err="failed to get container status \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": rpc error: code = NotFound desc = could not find container \"86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae\": container with ID starting with 86c2624331412c0515e644750b5cb5697b5bd85f31d836970ac32855fcdaf7ae not found: ID does not exist"
Jul 19 14:42:59.969395 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969391    2585 scope.go:117] "RemoveContainer" containerID="6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"
Jul 19 14:42:59.969665 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969647    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495"} err="failed to get container status \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": rpc error: code = NotFound desc = could not find container \"6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495\": container with ID starting with 6221ffe45cd2aa203f9dc22043add4c477cfb8b664bbc4b07a602dfcfa773495 not found: ID does not exist"
Jul 19 14:42:59.969721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969666    2585 scope.go:117] "RemoveContainer" containerID="10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"
Jul 19 14:42:59.969902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969880    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364"} err="failed to get container status \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": rpc error: code = NotFound desc = could not find container \"10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364\": container with ID starting with 10d25c15a4f3a909c9aeb7f996d6ba41a71f62ad427409428cd3481e2d542364 not found: ID does not exist"
Jul 19 14:42:59.969957 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.969903    2585 scope.go:117] "RemoveContainer" containerID="963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"
Jul 19 14:42:59.970134 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970109    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5"} err="failed to get container status \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": rpc error: code = NotFound desc = could not find container \"963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5\": container with ID starting with 963cda3a0bcd20fa6c90de51cf74cf294fea06098a4d76cdc4215ed686ef07d5 not found: ID does not exist"
Jul 19 14:42:59.970176 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970137    2585 scope.go:117] "RemoveContainer" containerID="92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"
Jul 19 14:42:59.970353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970329    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835"} err="failed to get container status \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": rpc error: code = NotFound desc = could not find container \"92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835\": container with ID starting with 92aead23f430526c9da64b8dd3460dbdcef8db69ce7538624389b67a4b619835 not found: ID does not exist"
Jul 19 14:42:59.970392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970357    2585 scope.go:117] "RemoveContainer" containerID="f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"
Jul 19 14:42:59.970608 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970584    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615"} err="failed to get container status \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": rpc error: code = NotFound desc = could not find container \"f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615\": container with ID starting with f91a7f3bd005d6dea3d4995e3a4f7b96b1b8f456cef8a8c7af02fdf55d616615 not found: ID does not exist"
Jul 19 14:42:59.970608 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970604    2585 scope.go:117] "RemoveContainer" containerID="9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"
Jul 19 14:42:59.970841 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.970821    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d"} err="failed to get container status \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": rpc error: code = NotFound desc = could not find container \"9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d\": container with ID starting with 9e2fc23a9d82feb23dcffc335720ad97b29ccc93655638a84b877a19381f105d not found: ID does not exist"
Jul 19 14:42:59.972522 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.972500    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 19 14:42:59.975980 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.975964    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 19 14:42:59.976212 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.976196    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 19 14:42:59.976820 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.976805    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 19 14:42:59.977107 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.977089    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7h33q5jn792ms\""
Jul 19 14:42:59.977279 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.977268    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vn4x6\""
Jul 19 14:42:59.978673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.978637    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 19 14:42:59.978673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.978647    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 19 14:42:59.978673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.978643    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 19 14:42:59.979109 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979092    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 19 14:42:59.979561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979547    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 19 14:42:59.979673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979548    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 19 14:42:59.979884 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979868    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 19 14:42:59.979982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979883    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 19 14:42:59.979982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.979908    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 19 14:42:59.980581 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.980562    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 19 14:42:59.988143 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.988119    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 19 14:42:59.995884 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:42:59.995858    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:43:00.022448 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022408    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022448 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022448    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022470    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztdl\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-kube-api-access-bztdl\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022504    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022540    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022562    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022591    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022618    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022649    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.022684 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022675    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022693    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022714    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022741    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022759    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022775    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022849    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022869    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022896    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.023017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.022912    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124261 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124219    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124271    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124296    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124321    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124342    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124368    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124393    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztdl\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-kube-api-access-bztdl\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124438    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124463    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124941 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124490    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.124981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.124967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125032 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125010    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125119 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125087    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125127    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125142    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125195    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125224    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125260    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125297    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125324    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.125731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.125458    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.126248    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.126341    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.126587    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.126944    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.127292    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.127459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.127360    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.128121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.128093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.128290 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.128266    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.128530 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.128476    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.128965 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.128942    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.129510 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.129478    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.129617 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.129597    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.129825 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.129801    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.130131 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.130105    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.130247 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.130231    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.130480 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.130464    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.130550 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.130534    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.134182 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.134165    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztdl\" (UniqueName: \"kubernetes.io/projected/6778a5d3-9fcf-4c96-bc82-91aba04bf72d-kube-api-access-bztdl\") pod \"prometheus-k8s-0\" (UID: \"6778a5d3-9fcf-4c96-bc82-91aba04bf72d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.278300 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.278259    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:00.422254 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.422216    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d5e9e5-9a18-4914-8206-945d77f70bca" path="/var/lib/kubelet/pods/77d5e9e5-9a18-4914-8206-945d77f70bca/volumes"
Jul 19 14:43:00.423477 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.423444    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 19 14:43:00.426827 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:43:00.426773    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6778a5d3_9fcf_4c96_bc82_91aba04bf72d.slice/crio-2f4a00397cbbb916b53bb05f05deb133d428fca564f7ac0b603f644c775c9be9 WatchSource:0}: Error finding container 2f4a00397cbbb916b53bb05f05deb133d428fca564f7ac0b603f644c775c9be9: Status 404 returned error can't find the container with id 2f4a00397cbbb916b53bb05f05deb133d428fca564f7ac0b603f644c775c9be9
Jul 19 14:43:00.910336 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.910239    2585 generic.go:356] "Generic (PLEG): container finished" podID="6778a5d3-9fcf-4c96-bc82-91aba04bf72d" containerID="774332ef3dcc7d1575c9db68d90e44fcf049bb1d675be349818fddc25cd8d756" exitCode=0
Jul 19 14:43:00.910336 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.910324    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerDied","Data":"774332ef3dcc7d1575c9db68d90e44fcf049bb1d675be349818fddc25cd8d756"}
Jul 19 14:43:00.910552 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:00.910404    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"2f4a00397cbbb916b53bb05f05deb133d428fca564f7ac0b603f644c775c9be9"}
Jul 19 14:43:01.918511 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918475    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"e32c5ed0d0308898c86a69357795283bf260bcfd7ce2c20bb0c6f710b0526840"}
Jul 19 14:43:01.918918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918570    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"4d8da3d1b29b183e3be193f1d10deb68e797dcca88fc4cfbf4c8184a4149afc4"}
Jul 19 14:43:01.918918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918598    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"a14b6e57691330285d5d59c961c45060c244037f498aee1df14e7b30b803ed72"}
Jul 19 14:43:01.918918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918623    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"9da9102c01cc855560e0ea63b4f7d9cf98594726bd5e3335344f348b9e576e4d"}
Jul 19 14:43:01.918918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918645    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"b6e306e6060750f17ca76fa9a914dad8b4ed16924a1e407174d95092ffed47ae"}
Jul 19 14:43:01.918918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.918666    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6778a5d3-9fcf-4c96-bc82-91aba04bf72d","Type":"ContainerStarted","Data":"c5cbb42cebf562743fef0c308a22d00926bdbc340d1f8caca48d86e60188dde5"}
Jul 19 14:43:01.947697 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:01.947283    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.927498135 podStartE2EDuration="2.947260634s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.762499ms" isStatefulPod=false podCreationTimestamp="2026-07-19 14:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:43:01.944861426 +0000 UTC m=+384.371338442" watchObservedRunningTime="2026-07-19 14:43:01.947260634 +0000 UTC m=+384.373737647"
Jul 19 14:43:02.680692 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:02.680651    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:02.732090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:02.732040    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:05.279212 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:05.279164    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:43:07.519927 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:07.519883    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:07.660438 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:07.660397    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:07.708071 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:07.708040    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:08.268532 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:08.268494    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:16.528732 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:16.528694    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:16.580816 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:16.580742    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:18.453143 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:18.453109    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:18.515179 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:18.515145    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:18.575512 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:18.575479    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:18.973262 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:18.973226    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:19.072813 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:19.072759    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:19.120964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:19.120921    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:43:41.269303 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:43:41.269267    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:00.278885 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:00.278778    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:44:00.294855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:00.294826    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:44:01.169218 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:01.169193    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 19 14:44:15.028377 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:15.028340    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:15.129607 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:15.129573    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:18.348316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:18.348280    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:18.401356 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:18.401326    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:21.059550 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:21.059504    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:30.725545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:30.725505    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:41.750029 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:41.749993    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:42.544930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:42.544894    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:45.361044 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:45.361012    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:45.415673 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:45.415622    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:47.154057 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:47.154019    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:47.261566 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:47.258096    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:47.664545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:47.664513    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:47.764382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:47.764348    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:50.052525 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:50.052478    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:50.460292 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:50.460204    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:52.361236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:52.361204    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:52.413293 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:52.413259    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:57.664942 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:57.664893    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:59.811212 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:59.811165    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:44:59.911392 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:44:59.911362    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:04.569080 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:04.569037    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:04.674886 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:04.674850    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:06.761233 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:06.761186    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:17.209237 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:17.209194    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:19.913108 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:19.913071    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:20.013344 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:20.013303    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:21.880206 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:21.880169    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:22.903424 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:22.903387    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:23.030289 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:23.030250    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:23.403487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:23.403452    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:23.521091 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:23.521050    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.202216 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.202180    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.420256 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.420221    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.544435 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.544400    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.583328 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.583291    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"]
Jul 19 14:45:24.586833 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.586815    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.589353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.589329    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 19 14:45:24.589486 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.589368    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vts5g\""
Jul 19 14:45:24.589611 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.589598    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 19 14:45:24.589827 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.589811    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 19 14:45:24.589933 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.589914    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 19 14:45:24.600451 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.600428    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"]
Jul 19 14:45:24.738753 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.738717    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw6w9\" (UniqueName: \"kubernetes.io/projected/104c28db-34da-4b5c-9cf7-b10e79e7035b-kube-api-access-fw6w9\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.738753 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.738764    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-manifests\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.739009 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.738823    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-tmp\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.739009 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.738920    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.739009 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.738945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.748855 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.748823    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.839708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.839597    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.839708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.839644    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.839708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.839683    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw6w9\" (UniqueName: \"kubernetes.io/projected/104c28db-34da-4b5c-9cf7-b10e79e7035b-kube-api-access-fw6w9\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.840050 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.839717    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-manifests\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.840050 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.839743    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-tmp\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.840165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.840132    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-manifests\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.842039 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.842011    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104c28db-34da-4b5c-9cf7-b10e79e7035b-tmp\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.842259 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.842242    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.842312 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.842298    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/104c28db-34da-4b5c-9cf7-b10e79e7035b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.853177 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.853134    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:24.854148 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.854121    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw6w9\" (UniqueName: \"kubernetes.io/projected/104c28db-34da-4b5c-9cf7-b10e79e7035b-kube-api-access-fw6w9\") pod \"opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6\" (UID: \"104c28db-34da-4b5c-9cf7-b10e79e7035b\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.897060 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.897016    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:24.948005 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:24.947969    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:25.040275 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:25.040236    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"]
Jul 19 14:45:25.042738 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:25.042709    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:25.043980 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:25.043957    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104c28db_34da_4b5c_9cf7_b10e79e7035b.slice/crio-f8fe79346644de3020a3fd5aaa66982326ee848b24bb669b44a4b65f66a63f24 WatchSource:0}: Error finding container f8fe79346644de3020a3fd5aaa66982326ee848b24bb669b44a4b65f66a63f24: Status 404 returned error can't find the container with id f8fe79346644de3020a3fd5aaa66982326ee848b24bb669b44a4b65f66a63f24
Jul 19 14:45:25.144561 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:25.144471    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:25.243895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:25.243859    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:25.470017 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:25.469925    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6" event={"ID":"104c28db-34da-4b5c-9cf7-b10e79e7035b","Type":"ContainerStarted","Data":"f8fe79346644de3020a3fd5aaa66982326ee848b24bb669b44a4b65f66a63f24"}
Jul 19 14:45:28.485542 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:28.485506    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6" event={"ID":"104c28db-34da-4b5c-9cf7-b10e79e7035b","Type":"ContainerStarted","Data":"8022b6d401877a75b433d7766cf1546be3641f69ed30d986c1f2f8f0a8d9f295"}
Jul 19 14:45:29.491333 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:29.491298    2585 generic.go:356] "Generic (PLEG): container finished" podID="104c28db-34da-4b5c-9cf7-b10e79e7035b" containerID="8022b6d401877a75b433d7766cf1546be3641f69ed30d986c1f2f8f0a8d9f295" exitCode=0
Jul 19 14:45:29.491747 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:29.491383    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6" event={"ID":"104c28db-34da-4b5c-9cf7-b10e79e7035b","Type":"ContainerDied","Data":"8022b6d401877a75b433d7766cf1546be3641f69ed30d986c1f2f8f0a8d9f295"}
Jul 19 14:45:30.497734 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:30.497697    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6" event={"ID":"104c28db-34da-4b5c-9cf7-b10e79e7035b","Type":"ContainerStarted","Data":"dbcc2810592f1887e7562ccb4f5de120a72832cb3d938ad3701efe4ec2c696b9"}
Jul 19 14:45:30.498208 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:30.497831    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:30.522509 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:30.522454    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6" podStartSLOduration=2.166159993 podStartE2EDuration="6.522435804s" totalImagesPullingTime="3.270099098s" totalInitContainerRuntime="1.086176713s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:45:30.521283427 +0000 UTC m=+532.947760442" watchObservedRunningTime="2026-07-19 14:45:30.522435804 +0000 UTC m=+532.948912890"
Jul 19 14:45:36.122404 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:36.122370    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:36.216909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:36.216867    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:41.505517 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:41.505486    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6"
Jul 19 14:45:44.418625 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:44.418571    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:44.535829 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:44.535764    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:44.992101 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:44.992056    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:45.026147 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.026101    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"]
Jul 19 14:45:45.038756 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.038728    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.041751 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.041720    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 19 14:45:45.041940 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.041816    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 19 14:45:45.041940 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.041878    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 19 14:45:45.042058 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.042039    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 19 14:45:45.043057 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.043029    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"]
Jul 19 14:45:45.106598 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.106556    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161ce75c-6d40-4d1c-8889-d327ee413665-tmp\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.106768 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.106616    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztb2\" (UniqueName: \"kubernetes.io/projected/161ce75c-6d40-4d1c-8889-d327ee413665-kube-api-access-bztb2\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.106768 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.106680    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161ce75c-6d40-4d1c-8889-d327ee413665-tls-certs\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.155940 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.155906    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:45.173848 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.173813    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qk9cl"]
Jul 19 14:45:45.177408 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.177387    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.179923 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.179900    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hpm6h\""
Jul 19 14:45:45.181150 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.181132    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 19 14:45:45.187674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.187642    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qk9cl"]
Jul 19 14:45:45.207264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.207230    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161ce75c-6d40-4d1c-8889-d327ee413665-tls-certs\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.207455 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.207276    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4t5v\" (UniqueName: \"kubernetes.io/projected/c00bb303-832b-484b-87f5-1ffe043c81d6-kube-api-access-w4t5v\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.207455 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.207301    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.207455 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.207388    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161ce75c-6d40-4d1c-8889-d327ee413665-tmp\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.207455 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.207432    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztb2\" (UniqueName: \"kubernetes.io/projected/161ce75c-6d40-4d1c-8889-d327ee413665-kube-api-access-bztb2\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.209741 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.209708    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/161ce75c-6d40-4d1c-8889-d327ee413665-tmp\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.209898 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.209881    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161ce75c-6d40-4d1c-8889-d327ee413665-tls-certs\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.216261 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.216235    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztb2\" (UniqueName: \"kubernetes.io/projected/161ce75c-6d40-4d1c-8889-d327ee413665-kube-api-access-bztb2\") pod \"kube-auth-proxy-6fc68cb667-699tn\" (UID: \"161ce75c-6d40-4d1c-8889-d327ee413665\") " pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.247433 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.247342    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:45.308459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.308426    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4t5v\" (UniqueName: \"kubernetes.io/projected/c00bb303-832b-484b-87f5-1ffe043c81d6-kube-api-access-w4t5v\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.308459 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.308465    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.308674 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:45.308590    2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 19 14:45:45.308674 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:45.308672    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert podName:c00bb303-832b-484b-87f5-1ffe043c81d6 nodeName:}" failed. No retries permitted until 2026-07-19 14:45:45.808651973 +0000 UTC m=+548.235128966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert") pod "odh-model-controller-7998c67dc4-qk9cl" (UID: "c00bb303-832b-484b-87f5-1ffe043c81d6") : secret "odh-model-controller-webhook-cert" not found
Jul 19 14:45:45.318072 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.318042    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4t5v\" (UniqueName: \"kubernetes.io/projected/c00bb303-832b-484b-87f5-1ffe043c81d6-kube-api-access-w4t5v\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.347599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.347564    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:45.349439 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.349419    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"
Jul 19 14:45:45.452948 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.452890    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:45.504930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.504904    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc68cb667-699tn"]
Jul 19 14:45:45.508046 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:45.508014    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161ce75c_6d40_4d1c_8889_d327ee413665.slice/crio-a8633eb0b5b8dc5558fac911ade991cbc944372c41fdd2b4e556f3640168ee0f WatchSource:0}: Error finding container a8633eb0b5b8dc5558fac911ade991cbc944372c41fdd2b4e556f3640168ee0f: Status 404 returned error can't find the container with id a8633eb0b5b8dc5558fac911ade991cbc944372c41fdd2b4e556f3640168ee0f
Jul 19 14:45:45.569129 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.569089    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn" event={"ID":"161ce75c-6d40-4d1c-8889-d327ee413665","Type":"ContainerStarted","Data":"a8633eb0b5b8dc5558fac911ade991cbc944372c41fdd2b4e556f3640168ee0f"}
Jul 19 14:45:45.813616 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.813504    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:45.813773 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:45.813654    2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 19 14:45:45.813773 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:45.813727    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert podName:c00bb303-832b-484b-87f5-1ffe043c81d6 nodeName:}" failed. No retries permitted until 2026-07-19 14:45:46.813709583 +0000 UTC m=+549.240186574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert") pod "odh-model-controller-7998c67dc4-qk9cl" (UID: "c00bb303-832b-484b-87f5-1ffe043c81d6") : secret "odh-model-controller-webhook-cert" not found
Jul 19 14:45:45.998753 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:45.998713    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:46.006649 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.006618    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:45:46.011238 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.011217    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.013434 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.013412    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 19 14:45:46.013434 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.013411    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 19 14:45:46.013614 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.013469    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 19 14:45:46.013698 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.013685    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 19 14:45:46.013813 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.013773    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kxhb7\""
Jul 19 14:45:46.021765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.021743    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:45:46.094489 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.094400    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:46.116363 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116312    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116535 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116420    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116535 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116535 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116722 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116586    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116722 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116634    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.116722 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.116677    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgjwz\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.158448 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.158405    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:46.215353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.215318    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:46.217573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217526    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.217761 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217649    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.217761 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217703    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgjwz\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.217946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217821    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.217946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217872    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.217946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.218169 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.217953    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.218709 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.218641    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.220604 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.220576    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.220732 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.220705    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.220870 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.220848    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.220978 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.220961    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.228397 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.228339    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgjwz\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.228397 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.228349    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token\") pod \"istiod-openshift-gateway-7574459656-49pr4\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.321468 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.321423    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:46.519824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.519765    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:45:46.522899 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:46.522868    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce5b251_f163_4f97_a778_8dcdd45d5e73.slice/crio-2621c4d129ab9e0c9e34a0f2997185c91a4c4a332b666507eee038c02facc2bf WatchSource:0}: Error finding container 2621c4d129ab9e0c9e34a0f2997185c91a4c4a332b666507eee038c02facc2bf: Status 404 returned error can't find the container with id 2621c4d129ab9e0c9e34a0f2997185c91a4c4a332b666507eee038c02facc2bf
Jul 19 14:45:46.576557 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.576511    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" event={"ID":"7ce5b251-f163-4f97-a778-8dcdd45d5e73","Type":"ContainerStarted","Data":"2621c4d129ab9e0c9e34a0f2997185c91a4c4a332b666507eee038c02facc2bf"}
Jul 19 14:45:46.824028 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.823989    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:46.826986 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.826955    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00bb303-832b-484b-87f5-1ffe043c81d6-cert\") pod \"odh-model-controller-7998c67dc4-qk9cl\" (UID: \"c00bb303-832b-484b-87f5-1ffe043c81d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:46.971043 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.971009    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:46.989045 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:46.989003    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:47.164432 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:47.163970    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qk9cl"]
Jul 19 14:45:47.167562 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:47.167524    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00bb303_832b_484b_87f5_1ffe043c81d6.slice/crio-6819eb1794122bebc3214cad33fb29f23878713e7340d6329f7afe01f72abc9e WatchSource:0}: Error finding container 6819eb1794122bebc3214cad33fb29f23878713e7340d6329f7afe01f72abc9e: Status 404 returned error can't find the container with id 6819eb1794122bebc3214cad33fb29f23878713e7340d6329f7afe01f72abc9e
Jul 19 14:45:47.585359 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:47.585280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl" event={"ID":"c00bb303-832b-484b-87f5-1ffe043c81d6","Type":"ContainerStarted","Data":"6819eb1794122bebc3214cad33fb29f23878713e7340d6329f7afe01f72abc9e"}
Jul 19 14:45:49.570657 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:49.570606    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:45:49.571053 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:49.570701    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:45:50.585157 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.585116    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:50.603430 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.603343    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn" event={"ID":"161ce75c-6d40-4d1c-8889-d327ee413665","Type":"ContainerStarted","Data":"f479ce5dc834d0f1ac9c2a28186f679aeb7172ca292e58300de7dee531e14f14"}
Jul 19 14:45:50.605387 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.605354    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" event={"ID":"7ce5b251-f163-4f97-a778-8dcdd45d5e73","Type":"ContainerStarted","Data":"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"}
Jul 19 14:45:50.605551 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.605505    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:50.607481 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.607456    2585 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-49pr4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 14:45:50.607591 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.607517    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 14:45:50.625088 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.625035    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fc68cb667-699tn" podStartSLOduration=2.624510611 podStartE2EDuration="6.625017162s" totalImagesPullingTime="4.000506551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:45:50.623385254 +0000 UTC m=+553.049862267" watchObservedRunningTime="2026-07-19 14:45:50.625017162 +0000 UTC m=+553.051494176"
Jul 19 14:45:50.650845 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.650764    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" podStartSLOduration=2.60631972 podStartE2EDuration="5.650740006s" totalImagesPullingTime="3.044420286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:45:50.649321851 +0000 UTC m=+553.075798865" watchObservedRunningTime="2026-07-19 14:45:50.650740006 +0000 UTC m=+553.077217020"
Jul 19 14:45:50.680276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:50.680241    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:51.136058 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:51.136017    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:51.315930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:51.315891    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:51.446312 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:51.446223    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:51.611902 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:51.611858    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:52.618082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:52.618048    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl" event={"ID":"c00bb303-832b-484b-87f5-1ffe043c81d6","Type":"ContainerStarted","Data":"22a98352fd9217f5c9725f633929851014066439d9abde2643fc0a6f4be3bd62"}
Jul 19 14:45:52.618599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:52.618147    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:45:52.633437 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:52.633383    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl" podStartSLOduration=2.816237797 podStartE2EDuration="7.633364792s" totalImagesPullingTime="4.817126995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:45:52.631891569 +0000 UTC m=+555.058368593" watchObservedRunningTime="2026-07-19 14:45:52.633364792 +0000 UTC m=+555.059841849"
Jul 19 14:45:56.773905 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.773869    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:56.871969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.871930    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:56.888172 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.888137    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"]
Jul 19 14:45:56.891903 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.891874    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:56.894513 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.894477    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 19 14:45:56.894687 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.894666    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-pgttv\""
Jul 19 14:45:56.901721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.901697    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"]
Jul 19 14:45:56.905118 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.905092    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:56.906477 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.906451    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"]
Jul 19 14:45:56.907739 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.907714    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 19 14:45:56.908264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.908241    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lb45h\""
Jul 19 14:45:56.918876 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.918847    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"]
Jul 19 14:45:56.950867 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:56.950830    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.025401 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.025304    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpd64\" (UniqueName: \"kubernetes.io/projected/f813f876-72df-41ae-b2cf-a3ad0f598b7b-kube-api-access-vpd64\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.025401 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.025348    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.025401 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.025374    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8de0306-00ee-4873-81ed-4306db0c034a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.025629 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.025514    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkkkp\" (UniqueName: \"kubernetes.io/projected/d8de0306-00ee-4873-81ed-4306db0c034a-kube-api-access-tkkkp\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.126066 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.125998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkkkp\" (UniqueName: \"kubernetes.io/projected/d8de0306-00ee-4873-81ed-4306db0c034a-kube-api-access-tkkkp\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.126248 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.126080    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpd64\" (UniqueName: \"kubernetes.io/projected/f813f876-72df-41ae-b2cf-a3ad0f598b7b-kube-api-access-vpd64\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.126248 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.126113    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.126248 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.126147    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8de0306-00ee-4873-81ed-4306db0c034a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.126425 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:57.126352    2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 19 14:45:57.126425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.126403    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.126524 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:57.126447    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert podName:f813f876-72df-41ae-b2cf-a3ad0f598b7b nodeName:}" failed. No retries permitted until 2026-07-19 14:45:57.626426853 +0000 UTC m=+560.052903865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert") pod "llmisvc-controller-manager-777c6bd7d5-5khfl" (UID: "f813f876-72df-41ae-b2cf-a3ad0f598b7b") : secret "llmisvc-webhook-server-cert" not found
Jul 19 14:45:57.128698 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.128676    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8de0306-00ee-4873-81ed-4306db0c034a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.142049 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.142025    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpd64\" (UniqueName: \"kubernetes.io/projected/f813f876-72df-41ae-b2cf-a3ad0f598b7b-kube-api-access-vpd64\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.142177 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.142084    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkkkp\" (UniqueName: \"kubernetes.io/projected/d8de0306-00ee-4873-81ed-4306db0c034a-kube-api-access-tkkkp\") pod \"kserve-controller-manager-d6f4b65b8-gmvds\" (UID: \"d8de0306-00ee-4873-81ed-4306db0c034a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.202338 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.202299    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:45:57.237517 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.237473    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.374120 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.374091    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"]
Jul 19 14:45:57.375144 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.375116    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:45:57.376610 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:57.376587    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8de0306_00ee_4873_81ed_4306db0c034a.slice/crio-2ec5e4a2780787467085ee2bfb817a65296bac43bcba54edfca387cebfb12e13 WatchSource:0}: Error finding container 2ec5e4a2780787467085ee2bfb817a65296bac43bcba54edfca387cebfb12e13: Status 404 returned error can't find the container with id 2ec5e4a2780787467085ee2bfb817a65296bac43bcba54edfca387cebfb12e13
Jul 19 14:45:57.630242 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.630149    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.632576 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.632557    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f813f876-72df-41ae-b2cf-a3ad0f598b7b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5khfl\" (UID: \"f813f876-72df-41ae-b2cf-a3ad0f598b7b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.640170 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.640140    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds" event={"ID":"d8de0306-00ee-4873-81ed-4306db0c034a","Type":"ContainerStarted","Data":"2ec5e4a2780787467085ee2bfb817a65296bac43bcba54edfca387cebfb12e13"}
Jul 19 14:45:57.817081 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.817037    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:45:57.964811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:57.964774    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"]
Jul 19 14:45:57.966040 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:45:57.966013    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf813f876_72df_41ae_b2cf_a3ad0f598b7b.slice/crio-03673e6964927afcb9037fbbc7fa5612d0e636f98daf44cff197a0d0b58faadf WatchSource:0}: Error finding container 03673e6964927afcb9037fbbc7fa5612d0e636f98daf44cff197a0d0b58faadf: Status 404 returned error can't find the container with id 03673e6964927afcb9037fbbc7fa5612d0e636f98daf44cff197a0d0b58faadf
Jul 19 14:45:58.042530 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.042488    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:45:58.042747 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.042721    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerName="discovery" containerID="cri-o://86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772" gracePeriod=30
Jul 19 14:45:58.326294 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.324981    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:58.438151 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438054    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438151 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438109    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438258    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438750 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438725    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438750 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438730    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:45:58.438954 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438832    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgjwz\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438954 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438869    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.438954 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.438933    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig\") pod \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\" (UID: \"7ce5b251-f163-4f97-a778-8dcdd45d5e73\") "
Jul 19 14:45:58.439421 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.439396    2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-ca-configmap\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.442309 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.442263    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:45:58.442545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.442513    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz" (OuterVolumeSpecName: "kube-api-access-rgjwz") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "kube-api-access-rgjwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:45:58.442625 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.442594    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts" (OuterVolumeSpecName: "cacerts") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:45:58.442930 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.442878    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:45:58.443080 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.443059    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs" (OuterVolumeSpecName: "local-certs") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:45:58.443192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.443172    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7ce5b251-f163-4f97-a778-8dcdd45d5e73" (UID: "7ce5b251-f163-4f97-a778-8dcdd45d5e73"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:45:58.539981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.539939    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.539981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.539969    2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce5b251-f163-4f97-a778-8dcdd45d5e73-local-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.539981 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.539985    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgjwz\" (UniqueName: \"kubernetes.io/projected/7ce5b251-f163-4f97-a778-8dcdd45d5e73-kube-api-access-rgjwz\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.540256 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.540001    2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-csr-dns-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.540256 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.540017    2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-istio-kubeconfig\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.540256 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.540030    2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ce5b251-f163-4f97-a778-8dcdd45d5e73-cacerts\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:45:58.646407 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.646375    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerID="86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772" exitCode=0
Jul 19 14:45:58.646582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.646423    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"
Jul 19 14:45:58.646582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.646461    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" event={"ID":"7ce5b251-f163-4f97-a778-8dcdd45d5e73","Type":"ContainerDied","Data":"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"}
Jul 19 14:45:58.646582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.646516    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-49pr4" event={"ID":"7ce5b251-f163-4f97-a778-8dcdd45d5e73","Type":"ContainerDied","Data":"2621c4d129ab9e0c9e34a0f2997185c91a4c4a332b666507eee038c02facc2bf"}
Jul 19 14:45:58.646582 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.646545    2585 scope.go:117] "RemoveContainer" containerID="86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"
Jul 19 14:45:58.647918 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.647897    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl" event={"ID":"f813f876-72df-41ae-b2cf-a3ad0f598b7b","Type":"ContainerStarted","Data":"03673e6964927afcb9037fbbc7fa5612d0e636f98daf44cff197a0d0b58faadf"}
Jul 19 14:45:58.655200 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.655178    2585 scope.go:117] "RemoveContainer" containerID="86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"
Jul 19 14:45:58.655484 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:45:58.655465    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772\": container with ID starting with 86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772 not found: ID does not exist" containerID="86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"
Jul 19 14:45:58.655544 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.655493    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772"} err="failed to get container status \"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772\": rpc error: code = NotFound desc = could not find container \"86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772\": container with ID starting with 86e97f6b27ff01ee96d5da6626823a4db5b315f19a95a1a1f3db400869f27772 not found: ID does not exist"
Jul 19 14:45:58.697550 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.697471    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:45:58.700716 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:45:58.700681    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-49pr4"]
Jul 19 14:46:00.422373 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:00.422336    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" path="/var/lib/kubelet/pods/7ce5b251-f163-4f97-a778-8dcdd45d5e73/volumes"
Jul 19 14:46:01.666720 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.666681    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl" event={"ID":"f813f876-72df-41ae-b2cf-a3ad0f598b7b","Type":"ContainerStarted","Data":"7df8ca7f292c845815d39333355f36fbda59c9fe2e4ce4f0a0eb5664e9c80de1"}
Jul 19 14:46:01.667222 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.666748    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:46:01.668202 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.668179    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds" event={"ID":"d8de0306-00ee-4873-81ed-4306db0c034a","Type":"ContainerStarted","Data":"c5f2596276aa42e4ea839742335ba9f24d15e96d8f3e8ef125c0ef0f8a940585"}
Jul 19 14:46:01.668334 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.668247    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:46:01.682179 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.682114    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl" podStartSLOduration=2.824992676 podStartE2EDuration="5.682097516s" totalImagesPullingTime="2.85710484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:01.681409063 +0000 UTC m=+564.107886101" watchObservedRunningTime="2026-07-19 14:46:01.682097516 +0000 UTC m=+564.108574575"
Jul 19 14:46:01.699049 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:01.698990    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds" podStartSLOduration=2.95348088 podStartE2EDuration="5.698973197s" totalImagesPullingTime="2.745492317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:01.698014638 +0000 UTC m=+564.124491651" watchObservedRunningTime="2026-07-19 14:46:01.698973197 +0000 UTC m=+564.125450209"
Jul 19 14:46:03.639230 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:03.639181    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qk9cl"
Jul 19 14:46:32.675558 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:32.675523    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5khfl"
Jul 19 14:46:32.678544 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:32.678524    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gmvds"
Jul 19 14:46:38.093235 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:38.093203    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:46:38.094244 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:38.094224    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:46:45.837564 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.837526    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:45.969264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.969230    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:45.969567 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.969555    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerName="discovery"
Jul 19 14:46:45.969659 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.969569    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerName="discovery"
Jul 19 14:46:45.969659 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.969641    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce5b251-f163-4f97-a778-8dcdd45d5e73" containerName="discovery"
Jul 19 14:46:45.972721 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.972697    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:45.975382 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.975353    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-fj6tw\""
Jul 19 14:46:45.975520 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.975495    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:46:45.976331 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.976312    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:46:45.982121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:45.982093    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:46.060084 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.060050    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdg9q\" (UniqueName: \"kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q\") pod \"kuadrant-operator-catalog-mltkw\" (UID: \"ccc2bc97-df11-4373-bfce-cf190abfc516\") " pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:46.160588 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.160509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdg9q\" (UniqueName: \"kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q\") pod \"kuadrant-operator-catalog-mltkw\" (UID: \"ccc2bc97-df11-4373-bfce-cf190abfc516\") " pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:46.168013 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.167983    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdg9q\" (UniqueName: \"kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q\") pod \"kuadrant-operator-catalog-mltkw\" (UID: \"ccc2bc97-df11-4373-bfce-cf190abfc516\") " pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:46.285202 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.285153    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:46.340888 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.340855    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:46.422989 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.422961    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:46.423138 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:46:46.423117    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc2bc97_df11_4373_bfce_cf190abfc516.slice/crio-80d76648abaa2ca6ff787b0a42e36e89d73fd724223232a69ffaba371d613d2c WatchSource:0}: Error finding container 80d76648abaa2ca6ff787b0a42e36e89d73fd724223232a69ffaba371d613d2c: Status 404 returned error can't find the container with id 80d76648abaa2ca6ff787b0a42e36e89d73fd724223232a69ffaba371d613d2c
Jul 19 14:46:46.424447 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.424429    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:46:46.548638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.548599    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frjtx"]
Jul 19 14:46:46.553139 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.553120    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:46.562524 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.562499    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frjtx"]
Jul 19 14:46:46.664228 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.664197    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bd4m\" (UniqueName: \"kubernetes.io/projected/40b44909-e9ff-4f93-8d34-2be2e7a4779e-kube-api-access-5bd4m\") pod \"kuadrant-operator-catalog-frjtx\" (UID: \"40b44909-e9ff-4f93-8d34-2be2e7a4779e\") " pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:46.764867 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.764833    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bd4m\" (UniqueName: \"kubernetes.io/projected/40b44909-e9ff-4f93-8d34-2be2e7a4779e-kube-api-access-5bd4m\") pod \"kuadrant-operator-catalog-frjtx\" (UID: \"40b44909-e9ff-4f93-8d34-2be2e7a4779e\") " pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:46.772244 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.772214    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bd4m\" (UniqueName: \"kubernetes.io/projected/40b44909-e9ff-4f93-8d34-2be2e7a4779e-kube-api-access-5bd4m\") pod \"kuadrant-operator-catalog-frjtx\" (UID: \"40b44909-e9ff-4f93-8d34-2be2e7a4779e\") " pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:46.862385 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.862350    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:46.865236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.865183    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:46.871728 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.871702    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" event={"ID":"ccc2bc97-df11-4373-bfce-cf190abfc516","Type":"ContainerStarted","Data":"80d76648abaa2ca6ff787b0a42e36e89d73fd724223232a69ffaba371d613d2c"}
Jul 19 14:46:46.982023 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:46.981972    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:46:47.014594 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:47.014516    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frjtx"]
Jul 19 14:46:47.017183 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:46:47.017137    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b44909_e9ff_4f93_8d34_2be2e7a4779e.slice/crio-8643b46c6bc522ab031f75f9230159e6b18c7efdb02d6901fdfe8c4aac817cf1 WatchSource:0}: Error finding container 8643b46c6bc522ab031f75f9230159e6b18c7efdb02d6901fdfe8c4aac817cf1: Status 404 returned error can't find the container with id 8643b46c6bc522ab031f75f9230159e6b18c7efdb02d6901fdfe8c4aac817cf1
Jul 19 14:46:47.877324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:47.877286    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frjtx" event={"ID":"40b44909-e9ff-4f93-8d34-2be2e7a4779e","Type":"ContainerStarted","Data":"8643b46c6bc522ab031f75f9230159e6b18c7efdb02d6901fdfe8c4aac817cf1"}
Jul 19 14:46:48.883750 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:48.883649    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" event={"ID":"ccc2bc97-df11-4373-bfce-cf190abfc516","Type":"ContainerStarted","Data":"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"}
Jul 19 14:46:48.884255 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:48.883736    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" podUID="ccc2bc97-df11-4373-bfce-cf190abfc516" containerName="registry-server" containerID="cri-o://44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8" gracePeriod=2
Jul 19 14:46:48.885341 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:48.885311    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frjtx" event={"ID":"40b44909-e9ff-4f93-8d34-2be2e7a4779e","Type":"ContainerStarted","Data":"da472c3b2014babdc1c3c52aa6ce411c9c150f38e6fe520c6fcd21cd88cb5692"}
Jul 19 14:46:48.903802 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:48.901536    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" podStartSLOduration=1.7129416050000001 podStartE2EDuration="3.901512593s" totalImagesPullingTime="2.188570988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:48.898740942 +0000 UTC m=+611.325217954" watchObservedRunningTime="2026-07-19 14:46:48.901512593 +0000 UTC m=+611.327989607"
Jul 19 14:46:48.917928 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:48.917870    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-frjtx" podStartSLOduration=1.322643215 podStartE2EDuration="2.917850585s" totalImagesPullingTime="1.59520737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:48.917031622 +0000 UTC m=+611.343508635" watchObservedRunningTime="2026-07-19 14:46:48.917850585 +0000 UTC m=+611.344327599"
Jul 19 14:46:49.131588 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.131561    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:49.286775 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.286734    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdg9q\" (UniqueName: \"kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q\") pod \"ccc2bc97-df11-4373-bfce-cf190abfc516\" (UID: \"ccc2bc97-df11-4373-bfce-cf190abfc516\") "
Jul 19 14:46:49.289005 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.288969    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q" (OuterVolumeSpecName: "kube-api-access-cdg9q") pod "ccc2bc97-df11-4373-bfce-cf190abfc516" (UID: "ccc2bc97-df11-4373-bfce-cf190abfc516"). InnerVolumeSpecName "kube-api-access-cdg9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:46:49.388138 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.388093    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdg9q\" (UniqueName: \"kubernetes.io/projected/ccc2bc97-df11-4373-bfce-cf190abfc516-kube-api-access-cdg9q\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:46:49.891890 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.891770    2585 generic.go:356] "Generic (PLEG): container finished" podID="ccc2bc97-df11-4373-bfce-cf190abfc516" containerID="44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8" exitCode=0
Jul 19 14:46:49.891890 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.891840    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mltkw"
Jul 19 14:46:49.891890 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.891854    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" event={"ID":"ccc2bc97-df11-4373-bfce-cf190abfc516","Type":"ContainerDied","Data":"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"}
Jul 19 14:46:49.892463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.891911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mltkw" event={"ID":"ccc2bc97-df11-4373-bfce-cf190abfc516","Type":"ContainerDied","Data":"80d76648abaa2ca6ff787b0a42e36e89d73fd724223232a69ffaba371d613d2c"}
Jul 19 14:46:49.892463 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.891932    2585 scope.go:117] "RemoveContainer" containerID="44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"
Jul 19 14:46:49.901433 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.901415    2585 scope.go:117] "RemoveContainer" containerID="44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"
Jul 19 14:46:49.901727 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:46:49.901711    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8\": container with ID starting with 44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8 not found: ID does not exist" containerID="44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"
Jul 19 14:46:49.901773 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.901737    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8"} err="failed to get container status \"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8\": rpc error: code = NotFound desc = could not find container \"44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8\": container with ID starting with 44ac69430ffce7ba47acbd443e13d5538bbd30b2ee055511354f902987a3aad8 not found: ID does not exist"
Jul 19 14:46:49.911942 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.911909    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:49.913632 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:49.913605    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mltkw"]
Jul 19 14:46:50.421556 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:50.421523    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc2bc97-df11-4373-bfce-cf190abfc516" path="/var/lib/kubelet/pods/ccc2bc97-df11-4373-bfce-cf190abfc516/volumes"
Jul 19 14:46:56.862747 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:56.862712    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:56.863210 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:56.862770    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:56.884944 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:56.884915    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:46:56.946079 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:46:56.946051    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-frjtx"
Jul 19 14:47:11.392198 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.392160    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:11.764165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.764128    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7987948c99-8vqv4"]
Jul 19 14:47:11.764552 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.764537    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc2bc97-df11-4373-bfce-cf190abfc516" containerName="registry-server"
Jul 19 14:47:11.764598 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.764554    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc2bc97-df11-4373-bfce-cf190abfc516" containerName="registry-server"
Jul 19 14:47:11.764632 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.764625    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc2bc97-df11-4373-bfce-cf190abfc516" containerName="registry-server"
Jul 19 14:47:11.767767 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.767745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.770036 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.770014    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 19 14:47:11.770913 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.770876    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 19 14:47:11.770913 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.770883    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 19 14:47:11.771090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.770941    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 19 14:47:11.771090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.770960    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 19 14:47:11.771090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.771023    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lzfm7\""
Jul 19 14:47:11.771090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.771052    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 19 14:47:11.771090 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.771087    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 19 14:47:11.775346 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.775326    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 19 14:47:11.779545 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.779522    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7987948c99-8vqv4"]
Jul 19 14:47:11.827984 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.827946    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:11.877980 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.877934    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878174 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878031    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-service-ca\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878174 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878062    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-trusted-ca-bundle\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878174 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878149    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878190    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4fx5\" (UniqueName: \"kubernetes.io/projected/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-kube-api-access-v4fx5\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-oauth-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.878316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.878292    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-oauth-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.927141 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.927105    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:11.979526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979484    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-service-ca\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-trusted-ca-bundle\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979582    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979616    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4fx5\" (UniqueName: \"kubernetes.io/projected/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-kube-api-access-v4fx5\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979643    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-oauth-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979667    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-oauth-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.979765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.979695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.980376 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.980352    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-service-ca\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.980511 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.980435    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.980573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.980523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-trusted-ca-bundle\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.980573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.980523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-oauth-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.982169 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.982146    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-serving-cert\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.982247 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.982192    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-console-oauth-config\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:11.990443 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:11.990419    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4fx5\" (UniqueName: \"kubernetes.io/projected/8b12484d-efb2-4a6b-9b69-77d3e451ed9c-kube-api-access-v4fx5\") pod \"console-7987948c99-8vqv4\" (UID: \"8b12484d-efb2-4a6b-9b69-77d3e451ed9c\") " pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:12.027818 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.027723    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:12.078617 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.078578    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:12.214521 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.214494    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7987948c99-8vqv4"]
Jul 19 14:47:12.217134 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:12.217097    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b12484d_efb2_4a6b_9b69_77d3e451ed9c.slice/crio-53b687b6b2319e3c5a43a2586529864f0438cc53226b614ace9eec2dc2e73630 WatchSource:0}: Error finding container 53b687b6b2319e3c5a43a2586529864f0438cc53226b614ace9eec2dc2e73630: Status 404 returned error can't find the container with id 53b687b6b2319e3c5a43a2586529864f0438cc53226b614ace9eec2dc2e73630
Jul 19 14:47:12.554627 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.554589    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:12.659678 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.659637    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:12.997963 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.997925    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7987948c99-8vqv4" event={"ID":"8b12484d-efb2-4a6b-9b69-77d3e451ed9c","Type":"ContainerStarted","Data":"f0141cad59b9cf20ded385f3186acea28292b68fe1cbe788f5033ae6ca7c16d3"}
Jul 19 14:47:12.998161 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:12.997981    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7987948c99-8vqv4" event={"ID":"8b12484d-efb2-4a6b-9b69-77d3e451ed9c","Type":"ContainerStarted","Data":"53b687b6b2319e3c5a43a2586529864f0438cc53226b614ace9eec2dc2e73630"}
Jul 19 14:47:13.016866 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:13.016810    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7987948c99-8vqv4" podStartSLOduration=2.016774666 podStartE2EDuration="2.016774666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:13.015038152 +0000 UTC m=+635.441515166" watchObservedRunningTime="2026-07-19 14:47:13.016774666 +0000 UTC m=+635.443251678"
Jul 19 14:47:15.231228 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:15.231197    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:16.505550 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.505511    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"]
Jul 19 14:47:16.509159 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.509139    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.511790 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.511766    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vgftl\""
Jul 19 14:47:16.522644 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.522618    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"]
Jul 19 14:47:16.602471 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.602436    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:16.620638 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.620592    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.620864 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.620657    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28m8\" (UniqueName: \"kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.721532 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.721490    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.721687 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.721544    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z28m8\" (UniqueName: \"kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.721936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.721914    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.740001 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.739972    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28m8\" (UniqueName: \"kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.819895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.819794    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:16.967839 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.966961    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"]
Jul 19 14:47:16.972827 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:16.972753    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d471e9_e2a8_46e2_a673_5e90921ddca1.slice/crio-5897f84cd7a72d76f5e368f908214fb66202c839dc0c324c3563959a927c353c WatchSource:0}: Error finding container 5897f84cd7a72d76f5e368f908214fb66202c839dc0c324c3563959a927c353c: Status 404 returned error can't find the container with id 5897f84cd7a72d76f5e368f908214fb66202c839dc0c324c3563959a927c353c
Jul 19 14:47:16.973297 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:16.973255    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:17.016165 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:17.016130    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr" event={"ID":"15d471e9-e2a8-46e2-a673-5e90921ddca1","Type":"ContainerStarted","Data":"5897f84cd7a72d76f5e368f908214fb66202c839dc0c324c3563959a927c353c"}
Jul 19 14:47:18.033649 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:18.033613    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:18.933137 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:18.933098    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"]
Jul 19 14:47:18.937109 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:18.937083    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:18.940199 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:18.940168    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-snlmz\""
Jul 19 14:47:18.955760 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:18.955724    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"]
Jul 19 14:47:19.043475 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.043439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkp9h\" (UniqueName: \"kubernetes.io/projected/0dc36132-4e13-41ff-874d-8b2b28a41197-kube-api-access-hkp9h\") pod \"authorino-operator-75cc8d5cb7-m9ctk\" (UID: \"0dc36132-4e13-41ff-874d-8b2b28a41197\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:19.144931 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.144878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkp9h\" (UniqueName: \"kubernetes.io/projected/0dc36132-4e13-41ff-874d-8b2b28a41197-kube-api-access-hkp9h\") pod \"authorino-operator-75cc8d5cb7-m9ctk\" (UID: \"0dc36132-4e13-41ff-874d-8b2b28a41197\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:19.154242 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.154205    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkp9h\" (UniqueName: \"kubernetes.io/projected/0dc36132-4e13-41ff-874d-8b2b28a41197-kube-api-access-hkp9h\") pod \"authorino-operator-75cc8d5cb7-m9ctk\" (UID: \"0dc36132-4e13-41ff-874d-8b2b28a41197\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:19.250010 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.249972    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:19.454683 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.454547    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"]
Jul 19 14:47:19.457303 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:19.457265    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc36132_4e13_41ff_874d_8b2b28a41197.slice/crio-6a6e68715e01618ddc0f023279adc167b781cad47dd71677cf60bbeb31f47d42 WatchSource:0}: Error finding container 6a6e68715e01618ddc0f023279adc167b781cad47dd71677cf60bbeb31f47d42: Status 404 returned error can't find the container with id 6a6e68715e01618ddc0f023279adc167b781cad47dd71677cf60bbeb31f47d42
Jul 19 14:47:19.599218 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.599133    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:19.688300 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:19.688264    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:20.033798 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:20.033742    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk" event={"ID":"0dc36132-4e13-41ff-874d-8b2b28a41197","Type":"ContainerStarted","Data":"6a6e68715e01618ddc0f023279adc167b781cad47dd71677cf60bbeb31f47d42"}
Jul 19 14:47:21.162624 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.162594    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"]
Jul 19 14:47:21.166530 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.166506    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:21.168915 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.168889    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4wnx8\""
Jul 19 14:47:21.169067 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.168889    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 19 14:47:21.176764 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.176394    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"]
Jul 19 14:47:21.265588 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.265546    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fks9r\" (UniqueName: \"kubernetes.io/projected/800dc795-fc6d-47f2-aa71-b45253f15c98-kube-api-access-fks9r\") pod \"dns-operator-controller-manager-864dd78c4c-xclsw\" (UID: \"800dc795-fc6d-47f2-aa71-b45253f15c98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:21.366821 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.366765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fks9r\" (UniqueName: \"kubernetes.io/projected/800dc795-fc6d-47f2-aa71-b45253f15c98-kube-api-access-fks9r\") pod \"dns-operator-controller-manager-864dd78c4c-xclsw\" (UID: \"800dc795-fc6d-47f2-aa71-b45253f15c98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:21.374755 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.374725    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fks9r\" (UniqueName: \"kubernetes.io/projected/800dc795-fc6d-47f2-aa71-b45253f15c98-kube-api-access-fks9r\") pod \"dns-operator-controller-manager-864dd78c4c-xclsw\" (UID: \"800dc795-fc6d-47f2-aa71-b45253f15c98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:21.481368 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.481330    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:21.788674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:21.788584    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:22.079599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:22.079503    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:22.079599 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:22.079572    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:22.085671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:22.085643    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:22.509931 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:22.509859    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"]
Jul 19 14:47:22.515631 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:22.515595    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800dc795_fc6d_47f2_aa71_b45253f15c98.slice/crio-4eb904278ce1f8b02b61bc164a80501ec1ac270a1fa56767bf9278c56400d505 WatchSource:0}: Error finding container 4eb904278ce1f8b02b61bc164a80501ec1ac270a1fa56767bf9278c56400d505: Status 404 returned error can't find the container with id 4eb904278ce1f8b02b61bc164a80501ec1ac270a1fa56767bf9278c56400d505
Jul 19 14:47:23.050529 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.050492    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr" event={"ID":"15d471e9-e2a8-46e2-a673-5e90921ddca1","Type":"ContainerStarted","Data":"518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a"}
Jul 19 14:47:23.050737 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.050559    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:23.052443 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.052416    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk" event={"ID":"0dc36132-4e13-41ff-874d-8b2b28a41197","Type":"ContainerStarted","Data":"e80194e84933ad4a8d80b0d4ca0613100325d8a3df69207ea6ec0b70e9450ddd"}
Jul 19 14:47:23.052554 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.052471    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:23.053860 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.053827    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw" event={"ID":"800dc795-fc6d-47f2-aa71-b45253f15c98","Type":"ContainerStarted","Data":"4eb904278ce1f8b02b61bc164a80501ec1ac270a1fa56767bf9278c56400d505"}
Jul 19 14:47:23.058192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.058171    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7987948c99-8vqv4"
Jul 19 14:47:23.074512 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.074451    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr" podStartSLOduration=1.693128881 podStartE2EDuration="7.074429748s" totalImagesPullingTime="5.381300867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:23.072382508 +0000 UTC m=+645.498859522" watchObservedRunningTime="2026-07-19 14:47:23.074429748 +0000 UTC m=+645.500906762"
Jul 19 14:47:23.108209 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.108142    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk" podStartSLOduration=2.170226953 podStartE2EDuration="5.108119634s" totalImagesPullingTime="2.937892681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:23.106397407 +0000 UTC m=+645.532874420" watchObservedRunningTime="2026-07-19 14:47:23.108119634 +0000 UTC m=+645.534596648"
Jul 19 14:47:23.237187 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.237146    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"]
Jul 19 14:47:23.246238 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.246205    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:23.251117 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.251085    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4hmn5\""
Jul 19 14:47:23.253536 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.253506    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"]
Jul 19 14:47:23.262870 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.262836    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.285083 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.285045    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8bgt\" (UniqueName: \"kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt\") pod \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" (UID: \"5578a413-e95d-4d71-a5fa-634062f1457d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:23.357306 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.357206    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.386537 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.386504    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bgt\" (UniqueName: \"kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt\") pod \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" (UID: \"5578a413-e95d-4d71-a5fa-634062f1457d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:23.398106 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.398080    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8bgt\" (UniqueName: \"kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt\") pod \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" (UID: \"5578a413-e95d-4d71-a5fa-634062f1457d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:23.461631 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.461592    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.564088 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.563767    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:23.617324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.617251    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:23.880726 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:23.880642    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"]
Jul 19 14:47:24.210694 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:24.210621    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5578a413_e95d_4d71_a5fa_634062f1457d.slice/crio-18a88bbed2f8aff4d708f224f960aaae3c92024bd8f1eb0df115adcfae438224 WatchSource:0}: Error finding container 18a88bbed2f8aff4d708f224f960aaae3c92024bd8f1eb0df115adcfae438224: Status 404 returned error can't find the container with id 18a88bbed2f8aff4d708f224f960aaae3c92024bd8f1eb0df115adcfae438224
Jul 19 14:47:25.068805 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.068594    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw" event={"ID":"800dc795-fc6d-47f2-aa71-b45253f15c98","Type":"ContainerStarted","Data":"afe3b9f618a6811da05d294ba181f5d08e7156474f3db7aeb64b842fde7d3b57"}
Jul 19 14:47:25.069342 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.068896    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:25.070371 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.070334    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x" event={"ID":"5578a413-e95d-4d71-a5fa-634062f1457d","Type":"ContainerStarted","Data":"18a88bbed2f8aff4d708f224f960aaae3c92024bd8f1eb0df115adcfae438224"}
Jul 19 14:47:25.100532 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.100479    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw" podStartSLOduration=2.357899662 podStartE2EDuration="4.100459203s" totalImagesPullingTime="1.742559541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:25.098456059 +0000 UTC m=+647.524933097" watchObservedRunningTime="2026-07-19 14:47:25.100459203 +0000 UTC m=+647.526936215"
Jul 19 14:47:25.384150 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.384051    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:25.480042 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.480003    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:25.547320 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.547281    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:25.668891 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:25.668803    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:26.076998 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:26.076960    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x" event={"ID":"5578a413-e95d-4d71-a5fa-634062f1457d","Type":"ContainerStarted","Data":"6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362"}
Jul 19 14:47:26.077503 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:26.077179    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:26.097162 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:26.097095    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x" podStartSLOduration=1.452897299 podStartE2EDuration="3.097077748s" totalImagesPullingTime="1.644180449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:26.095185032 +0000 UTC m=+648.521662047" watchObservedRunningTime="2026-07-19 14:47:26.097077748 +0000 UTC m=+648.523554761"
Jul 19 14:47:34.062436 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:34.062392    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:34.062909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:34.062472    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m9ctk"
Jul 19 14:47:34.664404 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:34.664347    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:34.795444 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:34.795411    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:35.822707 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.822670    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"]
Jul 19 14:47:35.823121 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.822861    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" containerName="manager" containerID="cri-o://518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a" gracePeriod=2
Jul 19 14:47:35.831731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.831699    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"]
Jul 19 14:47:35.832747 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.832705    2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"15d471e9-e2a8-46e2-a673-5e90921ddca1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:35.842264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.842235    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"]
Jul 19 14:47:35.842811 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.842488    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" containerName="manager" containerID="cri-o://6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362" gracePeriod=2
Jul 19 14:47:35.844438 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.844414    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:35.844938 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.844916    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" containerName="manager"
Jul 19 14:47:35.844938 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.844938    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" containerName="manager"
Jul 19 14:47:35.845085 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.845028    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" containerName="manager"
Jul 19 14:47:35.848276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.848258    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:35.848379 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.848362    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:35.855312 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.855268    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"]
Jul 19 14:47:35.864243 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.864210    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:35.864712 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.864675    2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5578a413-e95d-4d71-a5fa-634062f1457d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-zcr6x\": pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:35.867014 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.866980    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:35.867991 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.867971    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"]
Jul 19 14:47:35.868471 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.868451    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" containerName="manager"
Jul 19 14:47:35.868471 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.868474    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" containerName="manager"
Jul 19 14:47:35.868589 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.868582    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" containerName="manager"
Jul 19 14:47:35.869100 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.869067    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:35.871119 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.871092    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:35.871626 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.871611    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:35.882039 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:35.882005    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"]
Jul 19 14:47:35.901364 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.901317    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:35.910078 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:35.910035    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.011324 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.011291    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwv88\" (UniqueName: \"kubernetes.io/projected/57e5b952-9344-4a2f-8842-0f381ed35815-kube-api-access-gwv88\") pod \"limitador-operator-controller-manager-859d7b78bd-875f9\" (UID: \"57e5b952-9344-4a2f-8842-0f381ed35815\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:36.011465 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.011362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9pjb\" (UniqueName: \"kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.011465 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.011418    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.089691 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.089613    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xclsw"
Jul 19 14:47:36.092065 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.092023    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.093946 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.093906    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.111964 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.111935    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.112133 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.112016    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwv88\" (UniqueName: \"kubernetes.io/projected/57e5b952-9344-4a2f-8842-0f381ed35815-kube-api-access-gwv88\") pod \"limitador-operator-controller-manager-859d7b78bd-875f9\" (UID: \"57e5b952-9344-4a2f-8842-0f381ed35815\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:36.112133 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.112077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9pjb\" (UniqueName: \"kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.112449 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.112426    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.115094 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.115069    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.118328 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.118302    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.118908 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.118883    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.120815 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.120766    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.123156 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.123124    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.124946 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.124912    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.127337 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.127248    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9pjb\" (UniqueName: \"kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82kx9\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.127337 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.127321    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwv88\" (UniqueName: \"kubernetes.io/projected/57e5b952-9344-4a2f-8842-0f381ed35815-kube-api-access-gwv88\") pod \"limitador-operator-controller-manager-859d7b78bd-875f9\" (UID: \"57e5b952-9344-4a2f-8842-0f381ed35815\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:36.131802 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.131755    2585 generic.go:356] "Generic (PLEG): container finished" podID="15d471e9-e2a8-46e2-a673-5e90921ddca1" containerID="518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a" exitCode=0
Jul 19 14:47:36.131968 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.131948    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.132081 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.131951    2585 scope.go:117] "RemoveContainer" containerID="518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a"
Jul 19 14:47:36.134439 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.134299    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.135261 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.134893    2585 generic.go:356] "Generic (PLEG): container finished" podID="5578a413-e95d-4d71-a5fa-634062f1457d" containerID="6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362" exitCode=0
Jul 19 14:47:36.135261 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.134996    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.136250 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.136225    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.138333 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.138302    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zcr6x\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zcr6x"
Jul 19 14:47:36.140311 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.140279    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t5dr\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t5dr"
Jul 19 14:47:36.143158 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.143140    2585 scope.go:117] "RemoveContainer" containerID="518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a"
Jul 19 14:47:36.143522 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.143498    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a\": container with ID starting with 518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a not found: ID does not exist" containerID="518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a"
Jul 19 14:47:36.143573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.143534    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a"} err="failed to get container status \"518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a\": rpc error: code = NotFound desc = could not find container \"518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a\": container with ID starting with 518018d90928d8f5532ab367c857980c1c060262a7cf4595d63a423c41e8078a not found: ID does not exist"
Jul 19 14:47:36.143573 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.143557    2585 scope.go:117] "RemoveContainer" containerID="6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362"
Jul 19 14:47:36.152108 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.152088    2585 scope.go:117] "RemoveContainer" containerID="6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362"
Jul 19 14:47:36.152393 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:36.152373    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362\": container with ID starting with 6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362 not found: ID does not exist" containerID="6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362"
Jul 19 14:47:36.152458 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.152400    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362"} err="failed to get container status \"6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362\": rpc error: code = NotFound desc = could not find container \"6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362\": container with ID starting with 6c1ff5a6fc73329be90ac354e55ce082876eb3f44a2d1a8432f1238469411362 not found: ID does not exist"
Jul 19 14:47:36.212661 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.212622    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume\") pod \"15d471e9-e2a8-46e2-a673-5e90921ddca1\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") "
Jul 19 14:47:36.212879 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.212703    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z28m8\" (UniqueName: \"kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8\") pod \"15d471e9-e2a8-46e2-a673-5e90921ddca1\" (UID: \"15d471e9-e2a8-46e2-a673-5e90921ddca1\") "
Jul 19 14:47:36.212879 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.212734    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8bgt\" (UniqueName: \"kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt\") pod \"5578a413-e95d-4d71-a5fa-634062f1457d\" (UID: \"5578a413-e95d-4d71-a5fa-634062f1457d\") "
Jul 19 14:47:36.213276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.213247    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "15d471e9-e2a8-46e2-a673-5e90921ddca1" (UID: "15d471e9-e2a8-46e2-a673-5e90921ddca1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:47:36.214881 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.214857    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt" (OuterVolumeSpecName: "kube-api-access-k8bgt") pod "5578a413-e95d-4d71-a5fa-634062f1457d" (UID: "5578a413-e95d-4d71-a5fa-634062f1457d"). InnerVolumeSpecName "kube-api-access-k8bgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:47:36.214979 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.214862    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8" (OuterVolumeSpecName: "kube-api-access-z28m8") pod "15d471e9-e2a8-46e2-a673-5e90921ddca1" (UID: "15d471e9-e2a8-46e2-a673-5e90921ddca1"). InnerVolumeSpecName "kube-api-access-z28m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:47:36.262911 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.262872    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:36.269442 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.269421    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:36.314326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.314291    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z28m8\" (UniqueName: \"kubernetes.io/projected/15d471e9-e2a8-46e2-a673-5e90921ddca1-kube-api-access-z28m8\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:47:36.314326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.314322    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8bgt\" (UniqueName: \"kubernetes.io/projected/5578a413-e95d-4d71-a5fa-634062f1457d-kube-api-access-k8bgt\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:47:36.314326 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.314334    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d471e9-e2a8-46e2-a673-5e90921ddca1-extensions-socket-volume\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:47:36.420868 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:36.420837    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438000f2_1992_405c_9589_bbb0eaee24c8.slice/crio-9a951b556b3239f34443cd76bd73ac178415db6483ba72a1dc23f36084e68caa WatchSource:0}: Error finding container 9a951b556b3239f34443cd76bd73ac178415db6483ba72a1dc23f36084e68caa: Status 404 returned error can't find the container with id 9a951b556b3239f34443cd76bd73ac178415db6483ba72a1dc23f36084e68caa
Jul 19 14:47:36.422518 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.422489    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d471e9-e2a8-46e2-a673-5e90921ddca1" path="/var/lib/kubelet/pods/15d471e9-e2a8-46e2-a673-5e90921ddca1/volumes"
Jul 19 14:47:36.422921 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.422895    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5578a413-e95d-4d71-a5fa-634062f1457d" path="/var/lib/kubelet/pods/5578a413-e95d-4d71-a5fa-634062f1457d/volumes"
Jul 19 14:47:36.423289 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.423272    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:36.440735 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.440704    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"]
Jul 19 14:47:36.444820 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:36.444747    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e5b952_9344_4a2f_8842_0f381ed35815.slice/crio-2b91c4e4f0bd900e39552765d390130edfc661bff1e43c8262f46d6601b331c9 WatchSource:0}: Error finding container 2b91c4e4f0bd900e39552765d390130edfc661bff1e43c8262f46d6601b331c9: Status 404 returned error can't find the container with id 2b91c4e4f0bd900e39552765d390130edfc661bff1e43c8262f46d6601b331c9
Jul 19 14:47:36.736695 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.736665    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:47:36.740364 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.740325    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.754433 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.754406    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:47:36.819983 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.819945    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgct6\" (UniqueName: \"kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.819983 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.819988    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.920829 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.920755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgct6\" (UniqueName: \"kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.921240 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.920845    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.921240 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.921202    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:36.934426 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:36.934398    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgct6\" (UniqueName: \"kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6\") pod \"kuadrant-operator-controller-manager-86878c769f-458lz\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:37.051681 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.051588    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:37.102062 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.101878    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:37.146837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.146725    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9" event={"ID":"57e5b952-9344-4a2f-8842-0f381ed35815","Type":"ContainerStarted","Data":"4557d3f03a32294ae1c2aa2b114814a0d98079116ff9fb9f572c83cd36b13924"}
Jul 19 14:47:37.146837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.146811    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:37.146837 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.146840    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9" event={"ID":"57e5b952-9344-4a2f-8842-0f381ed35815","Type":"ContainerStarted","Data":"2b91c4e4f0bd900e39552765d390130edfc661bff1e43c8262f46d6601b331c9"}
Jul 19 14:47:37.150124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.149979    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" event={"ID":"438000f2-1992-405c-9589-bbb0eaee24c8","Type":"ContainerStarted","Data":"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"}
Jul 19 14:47:37.150124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.150032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" event={"ID":"438000f2-1992-405c-9589-bbb0eaee24c8","Type":"ContainerStarted","Data":"9a951b556b3239f34443cd76bd73ac178415db6483ba72a1dc23f36084e68caa"}
Jul 19 14:47:37.150124 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.150076    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:37.167414 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.167352    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9" podStartSLOduration=2.167311816 podStartE2EDuration="2.167311816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:37.165958134 +0000 UTC m=+659.592435188" watchObservedRunningTime="2026-07-19 14:47:37.167311816 +0000 UTC m=+659.593789046"
Jul 19 14:47:37.171577 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.171553    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:37.189262 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.189206    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" podStartSLOduration=2.189186484 podStartE2EDuration="2.189186484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:37.187467588 +0000 UTC m=+659.613944614" watchObservedRunningTime="2026-07-19 14:47:37.189186484 +0000 UTC m=+659.615663496"
Jul 19 14:47:37.208931 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:37.208905    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:47:37.211533 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:47:37.211496    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c73b458_2fc2_40b0_8e7c_1f2c9bef98d2.slice/crio-e0068fdb07921b22ddb44fe77d22336124a091808143c7f5bb06a299a523dfe2 WatchSource:0}: Error finding container e0068fdb07921b22ddb44fe77d22336124a091808143c7f5bb06a299a523dfe2: Status 404 returned error can't find the container with id e0068fdb07921b22ddb44fe77d22336124a091808143c7f5bb06a299a523dfe2
Jul 19 14:47:38.157999 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:38.157960    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" event={"ID":"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2","Type":"ContainerStarted","Data":"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"}
Jul 19 14:47:38.158466 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:38.158026    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" event={"ID":"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2","Type":"ContainerStarted","Data":"e0068fdb07921b22ddb44fe77d22336124a091808143c7f5bb06a299a523dfe2"}
Jul 19 14:47:38.158466 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:38.158266    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:38.182317 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:38.182259    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" podStartSLOduration=2.182237605 podStartE2EDuration="2.182237605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:38.180450694 +0000 UTC m=+660.606927708" watchObservedRunningTime="2026-07-19 14:47:38.182237605 +0000 UTC m=+660.608714619"
Jul 19 14:47:41.796988 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:41.796954    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:41.899982 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:41.899948    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:45.605586 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:45.605555    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:45.664252 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:45.664219    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:48.160494 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:48.160461    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-875f9"
Jul 19 14:47:48.160973 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:48.160531    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:49.166057 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.166029    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:47:49.222277 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.222220    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:49.222699 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.222648    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" podUID="438000f2-1992-405c-9589-bbb0eaee24c8" containerName="manager" containerID="cri-o://a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a" gracePeriod=10
Jul 19 14:47:49.478322 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.478295    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:49.533156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.533121    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9pjb\" (UniqueName: \"kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb\") pod \"438000f2-1992-405c-9589-bbb0eaee24c8\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") "
Jul 19 14:47:49.533156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.533164    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume\") pod \"438000f2-1992-405c-9589-bbb0eaee24c8\" (UID: \"438000f2-1992-405c-9589-bbb0eaee24c8\") "
Jul 19 14:47:49.533551 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.533528    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "438000f2-1992-405c-9589-bbb0eaee24c8" (UID: "438000f2-1992-405c-9589-bbb0eaee24c8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:47:49.535261 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.535236    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb" (OuterVolumeSpecName: "kube-api-access-t9pjb") pod "438000f2-1992-405c-9589-bbb0eaee24c8" (UID: "438000f2-1992-405c-9589-bbb0eaee24c8"). InnerVolumeSpecName "kube-api-access-t9pjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:47:49.633639 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.633597    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9pjb\" (UniqueName: \"kubernetes.io/projected/438000f2-1992-405c-9589-bbb0eaee24c8-kube-api-access-t9pjb\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:47:49.633639 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:49.633629    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/438000f2-1992-405c-9589-bbb0eaee24c8-extensions-socket-volume\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:47:50.231236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.231195    2585 generic.go:356] "Generic (PLEG): container finished" podID="438000f2-1992-405c-9589-bbb0eaee24c8" containerID="a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a" exitCode=0
Jul 19 14:47:50.231236 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.231238    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"
Jul 19 14:47:50.231680 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.231280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" event={"ID":"438000f2-1992-405c-9589-bbb0eaee24c8","Type":"ContainerDied","Data":"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"}
Jul 19 14:47:50.231680 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.231333    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9" event={"ID":"438000f2-1992-405c-9589-bbb0eaee24c8","Type":"ContainerDied","Data":"9a951b556b3239f34443cd76bd73ac178415db6483ba72a1dc23f36084e68caa"}
Jul 19 14:47:50.231680 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.231355    2585 scope.go:117] "RemoveContainer" containerID="a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"
Jul 19 14:47:50.240176 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.240160    2585 scope.go:117] "RemoveContainer" containerID="a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"
Jul 19 14:47:50.240421 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:47:50.240405    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a\": container with ID starting with a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a not found: ID does not exist" containerID="a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"
Jul 19 14:47:50.240473 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.240428    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a"} err="failed to get container status \"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a\": rpc error: code = NotFound desc = could not find container \"a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a\": container with ID starting with a7de2168030d0b1a78b6670806c1408bf3292cb42e66d23208e55115bf17c88a not found: ID does not exist"
Jul 19 14:47:50.255227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.255190    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:50.261494 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.261466    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82kx9"]
Jul 19 14:47:50.422939 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:50.422907    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="438000f2-1992-405c-9589-bbb0eaee24c8" path="/var/lib/kubelet/pods/438000f2-1992-405c-9589-bbb0eaee24c8/volumes"
Jul 19 14:47:54.346872 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:54.346839    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:54.434088 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:54.434050    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:56.750733 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:56.750688    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:56.804011 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:47:56.803976    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:01.499666 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:01.499625    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:01.605427 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:01.605394    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:05.520397 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.520354    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:05.596280 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.596243    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"]
Jul 19 14:48:05.596719 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.596698    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="438000f2-1992-405c-9589-bbb0eaee24c8" containerName="manager"
Jul 19 14:48:05.596719 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.596718    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="438000f2-1992-405c-9589-bbb0eaee24c8" containerName="manager"
Jul 19 14:48:05.596938 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.596802    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="438000f2-1992-405c-9589-bbb0eaee24c8" containerName="manager"
Jul 19 14:48:05.600112 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.600090    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.602541 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.602519    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gzsp9\""
Jul 19 14:48:05.602658 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.602547    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 19 14:48:05.609841 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.609811    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"]
Jul 19 14:48:05.619769 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.619736    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:05.665347 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665300    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-442wx\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-kube-api-access-442wx\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665464    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665498    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54f417ba-7683-404b-add7-d30b4d19f264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665533 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665516    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665570    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665593    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54f417ba-7683-404b-add7-d30b4d19f264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665671 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665640    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.665775 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.665673    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.680345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.680314    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:05.766368 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766445    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54f417ba-7683-404b-add7-d30b4d19f264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766469    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54f417ba-7683-404b-add7-d30b4d19f264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766601 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766580    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766609    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766898    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766914    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.766969 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.766957    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-442wx\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-kube-api-access-442wx\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.767207 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.767009    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.767281 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.767238    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.767423 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.767403    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54f417ba-7683-404b-add7-d30b4d19f264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.769552 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.769520    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54f417ba-7683-404b-add7-d30b4d19f264-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.769702 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.769684    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54f417ba-7683-404b-add7-d30b4d19f264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.778524 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.778466    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.778626 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.778551    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-442wx\" (UniqueName: \"kubernetes.io/projected/54f417ba-7683-404b-add7-d30b4d19f264-kube-api-access-442wx\") pod \"maas-default-gateway-openshift-default-56654dcf7-lrqb5\" (UID: \"54f417ba-7683-404b-add7-d30b4d19f264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:05.911496 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:05.911463    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:06.059970 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:06.059939    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:06.060579 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:06.060559    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"]
Jul 19 14:48:06.062766 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:48:06.062736    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f417ba_7683_404b_add7_d30b4d19f264.slice/crio-3cc9ec0eb248f3bf2802c1ad876f00a70336347234685461a8e5144b54089efd WatchSource:0}: Error finding container 3cc9ec0eb248f3bf2802c1ad876f00a70336347234685461a8e5144b54089efd: Status 404 returned error can't find the container with id 3cc9ec0eb248f3bf2802c1ad876f00a70336347234685461a8e5144b54089efd
Jul 19 14:48:06.316689 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:06.316597    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" event={"ID":"54f417ba-7683-404b-add7-d30b4d19f264","Type":"ContainerStarted","Data":"3cc9ec0eb248f3bf2802c1ad876f00a70336347234685461a8e5144b54089efd"}
Jul 19 14:48:08.758068 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:08.758032    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:48:08.758353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:08.758106    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:48:08.758353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:08.758137    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:48:09.335455 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:09.335421    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" event={"ID":"54f417ba-7683-404b-add7-d30b4d19f264","Type":"ContainerStarted","Data":"e889b9a24e563f5778bac998b9f7b9271f9037642ca10c62b1119c623e6e4523"}
Jul 19 14:48:09.356831 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:09.356763    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" podStartSLOduration=1.6642962209999999 podStartE2EDuration="4.356747677s" totalImagesPullingTime="2.692451456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:09.35413223 +0000 UTC m=+691.780609245" watchObservedRunningTime="2026-07-19 14:48:09.356747677 +0000 UTC m=+691.783224689"
Jul 19 14:48:09.911676 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:09.911637    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:09.916818 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:09.916772    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:10.341425 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.341389    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:10.342410 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.342381    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:48:10.450028 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.449994    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:10.470356 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.470317    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:10.474022 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.473998    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.476886 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.476863    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nh8pg\""
Jul 19 14:48:10.477004 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.476909    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 19 14:48:10.483268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.483242    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:10.511925 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.511894    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.512351 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.512330    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqv49\" (UniqueName: \"kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.548278 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.548209    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:10.564714 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.564665    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:10.613303 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.613205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.613303 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.613295    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqv49\" (UniqueName: \"kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.613973 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.613948    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.621003 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.620972    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqv49\" (UniqueName: \"kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49\") pod \"limitador-limitador-64c94fd64f-klpgg\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.650596 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.650554    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:10.785951 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.785914    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:10.934452 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:10.934424    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:10.937561 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:48:10.937503    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebcdeb9_a333_40cc_9983_2ea9129faf31.slice/crio-1703d1c377659b11958957561e95156fb2aafa87260595199fa36dec802d85fa WatchSource:0}: Error finding container 1703d1c377659b11958957561e95156fb2aafa87260595199fa36dec802d85fa: Status 404 returned error can't find the container with id 1703d1c377659b11958957561e95156fb2aafa87260595199fa36dec802d85fa
Jul 19 14:48:11.282735 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.282682    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:11.347350 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.347316    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" event={"ID":"9ebcdeb9-a333-40cc-9983-2ea9129faf31","Type":"ContainerStarted","Data":"1703d1c377659b11958957561e95156fb2aafa87260595199fa36dec802d85fa"}
Jul 19 14:48:11.383095 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.383057    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:11.464799 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.464752    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:11.469769 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.469740    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:11.471913 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.471887    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-k9sdp\""
Jul 19 14:48:11.478015 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.477990    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:11.522064 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.522034    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27vm\" (UniqueName: \"kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm\") pod \"authorino-75d556bd77-grcdw\" (UID: \"7caf3e27-e37f-400e-b947-faf7874cad0a\") " pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:11.623526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.623280    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27vm\" (UniqueName: \"kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm\") pod \"authorino-75d556bd77-grcdw\" (UID: \"7caf3e27-e37f-400e-b947-faf7874cad0a\") " pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:11.631515 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.631482    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27vm\" (UniqueName: \"kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm\") pod \"authorino-75d556bd77-grcdw\" (UID: \"7caf3e27-e37f-400e-b947-faf7874cad0a\") " pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:11.780689 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.780647    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:11.979319 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:11.979282    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:11.981467 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:48:11.981436    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7caf3e27_e37f_400e_b947_faf7874cad0a.slice/crio-9e5490178c063f6f327de6d4922993f826f4110c5bc1678c34ab472c230cece8 WatchSource:0}: Error finding container 9e5490178c063f6f327de6d4922993f826f4110c5bc1678c34ab472c230cece8: Status 404 returned error can't find the container with id 9e5490178c063f6f327de6d4922993f826f4110c5bc1678c34ab472c230cece8
Jul 19 14:48:12.356316 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:12.356243    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-grcdw" event={"ID":"7caf3e27-e37f-400e-b947-faf7874cad0a","Type":"ContainerStarted","Data":"9e5490178c063f6f327de6d4922993f826f4110c5bc1678c34ab472c230cece8"}
Jul 19 14:48:15.702114 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:15.702077    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:15.806211 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:15.806163    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:15.901695 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:15.901606    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:16.002746 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.002712    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:16.390482 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.390440    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-grcdw" event={"ID":"7caf3e27-e37f-400e-b947-faf7874cad0a","Type":"ContainerStarted","Data":"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"}
Jul 19 14:48:16.392097 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.392069    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" event={"ID":"9ebcdeb9-a333-40cc-9983-2ea9129faf31","Type":"ContainerStarted","Data":"793a19b74351c3b3a280f84705d30f46e9076cf0881e06735ef34e023dc78467"}
Jul 19 14:48:16.392258 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.392156    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:16.405895 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.405827    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-grcdw" podStartSLOduration=1.775778266 podStartE2EDuration="5.405804618s" totalImagesPullingTime="3.630026352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:16.40498509 +0000 UTC m=+698.831462104" watchObservedRunningTime="2026-07-19 14:48:16.405804618 +0000 UTC m=+698.832281633"
Jul 19 14:48:16.421277 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:16.421213    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" podStartSLOduration=1.754016236 podStartE2EDuration="6.421192603s" totalImagesPullingTime="4.667176367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:16.420291243 +0000 UTC m=+698.846768257" watchObservedRunningTime="2026-07-19 14:48:16.421192603 +0000 UTC m=+698.847669618"
Jul 19 14:48:17.500076 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:17.500038    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:17.600973 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:17.600934    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:18.506882 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:18.506846    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:18.606351 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:18.606317    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:21.910726 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:21.910689    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:22.014149 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:22.014111    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:26.939348 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:26.939304    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:26.939828 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:26.939549    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" podUID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" containerName="limitador" containerID="cri-o://793a19b74351c3b3a280f84705d30f46e9076cf0881e06735ef34e023dc78467" gracePeriod=30
Jul 19 14:48:26.940516 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:26.940482    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:27.455024 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.454988    2585 generic.go:356] "Generic (PLEG): container finished" podID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" containerID="793a19b74351c3b3a280f84705d30f46e9076cf0881e06735ef34e023dc78467" exitCode=0
Jul 19 14:48:27.455197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.455056    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" event={"ID":"9ebcdeb9-a333-40cc-9983-2ea9129faf31","Type":"ContainerDied","Data":"793a19b74351c3b3a280f84705d30f46e9076cf0881e06735ef34e023dc78467"}
Jul 19 14:48:27.505821 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.505737    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:27.571353 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.571319    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqv49\" (UniqueName: \"kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49\") pod \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") "
Jul 19 14:48:27.571566 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.571406    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file\") pod \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\" (UID: \"9ebcdeb9-a333-40cc-9983-2ea9129faf31\") "
Jul 19 14:48:27.571824 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.571771    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file" (OuterVolumeSpecName: "config-file") pod "9ebcdeb9-a333-40cc-9983-2ea9129faf31" (UID: "9ebcdeb9-a333-40cc-9983-2ea9129faf31"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:48:27.573449 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.573418    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49" (OuterVolumeSpecName: "kube-api-access-xqv49") pod "9ebcdeb9-a333-40cc-9983-2ea9129faf31" (UID: "9ebcdeb9-a333-40cc-9983-2ea9129faf31"). InnerVolumeSpecName "kube-api-access-xqv49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:48:27.672244 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.672202    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqv49\" (UniqueName: \"kubernetes.io/projected/9ebcdeb9-a333-40cc-9983-2ea9129faf31-kube-api-access-xqv49\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:48:27.672244 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:27.672235    2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ebcdeb9-a333-40cc-9983-2ea9129faf31-config-file\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:48:28.461516 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:28.461490    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg"
Jul 19 14:48:28.461516 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:28.461502    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-klpgg" event={"ID":"9ebcdeb9-a333-40cc-9983-2ea9129faf31","Type":"ContainerDied","Data":"1703d1c377659b11958957561e95156fb2aafa87260595199fa36dec802d85fa"}
Jul 19 14:48:28.462022 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:28.461564    2585 scope.go:117] "RemoveContainer" containerID="793a19b74351c3b3a280f84705d30f46e9076cf0881e06735ef34e023dc78467"
Jul 19 14:48:28.478026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:28.477989    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:28.479799 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:28.479765    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-klpgg"]
Jul 19 14:48:30.422650 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:30.422569    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" path="/var/lib/kubelet/pods/9ebcdeb9-a333-40cc-9983-2ea9129faf31/volumes"
Jul 19 14:48:31.655172 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:31.655141    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:31.761102 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:31.761037    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:32.125227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.117107    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"]
Jul 19 14:48:32.125227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.118160    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" containerName="limitador"
Jul 19 14:48:32.125227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.118181    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" containerName="limitador"
Jul 19 14:48:32.125227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.118353    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebcdeb9-a333-40cc-9983-2ea9129faf31" containerName="limitador"
Jul 19 14:48:32.127710 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.127676    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.129845 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.129745    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"]
Jul 19 14:48:32.130297 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.130274    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5k4z4\""
Jul 19 14:48:32.130454 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.130428    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 19 14:48:32.131111 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.131089    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 14:48:32.131238 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.131098    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 19 14:48:32.155708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.155666    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:32.216106 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.216068    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsxvv\" (UniqueName: \"kubernetes.io/projected/b95e9338-b3fb-48d2-aae8-d96dd952a32a-kube-api-access-lsxvv\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.216268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.216124    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b95e9338-b3fb-48d2-aae8-d96dd952a32a-data\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.317403 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.317357    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsxvv\" (UniqueName: \"kubernetes.io/projected/b95e9338-b3fb-48d2-aae8-d96dd952a32a-kube-api-access-lsxvv\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.317590 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.317416    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b95e9338-b3fb-48d2-aae8-d96dd952a32a-data\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.317766 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.317749    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b95e9338-b3fb-48d2-aae8-d96dd952a32a-data\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.325457 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.325431    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsxvv\" (UniqueName: \"kubernetes.io/projected/b95e9338-b3fb-48d2-aae8-d96dd952a32a-kube-api-access-lsxvv\") pod \"postgres-69fb4594cf-cvw9l\" (UID: \"b95e9338-b3fb-48d2-aae8-d96dd952a32a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.439623 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.439526    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:32.596231 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:32.596202    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"]
Jul 19 14:48:32.598992 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:48:32.598962    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95e9338_b3fb_48d2_aae8_d96dd952a32a.slice/crio-a4f61b9673f8a4bdd1f37419768e1cf3b7d8e032c87a5a537e3436780d25ccf9 WatchSource:0}: Error finding container a4f61b9673f8a4bdd1f37419768e1cf3b7d8e032c87a5a537e3436780d25ccf9: Status 404 returned error can't find the container with id a4f61b9673f8a4bdd1f37419768e1cf3b7d8e032c87a5a537e3436780d25ccf9
Jul 19 14:48:33.494138 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:33.494089    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l" event={"ID":"b95e9338-b3fb-48d2-aae8-d96dd952a32a","Type":"ContainerStarted","Data":"a4f61b9673f8a4bdd1f37419768e1cf3b7d8e032c87a5a537e3436780d25ccf9"}
Jul 19 14:48:37.518212 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:37.518178    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l" event={"ID":"b95e9338-b3fb-48d2-aae8-d96dd952a32a","Type":"ContainerStarted","Data":"72e7f3e33af783c8c12225e1d57db27cfc67dd151f1d4ba74fe80b170ba90870"}
Jul 19 14:48:37.518619 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:37.518284    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:37.531608 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:37.531550    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l" podStartSLOduration=0.923946038 podStartE2EDuration="5.531531829s" totalImagesPullingTime="4.607585791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:37.530421581 +0000 UTC m=+719.956898596" watchObservedRunningTime="2026-07-19 14:48:37.531531829 +0000 UTC m=+719.958008842"
Jul 19 14:48:43.554523 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:43.554489    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cvw9l"
Jul 19 14:48:43.882889 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:43.882795    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:44.213395 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.213309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:44.450062 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.450021    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:48:44.456128 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.456098    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.458391 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.458366    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 19 14:48:44.461833 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.461794    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:48:44.534839 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.534803    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj68c\" (UniqueName: \"kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.535026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.534884    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.635909 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.635870    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj68c\" (UniqueName: \"kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.636401 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.635946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.638595 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.638569    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.643742 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.643719    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj68c\" (UniqueName: \"kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c\") pod \"authorino-cb766b755-6bk77\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") " pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.768487 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.768439    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:48:44.909420 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:44.909349    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:48:44.912074 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:48:44.912048    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd17ad151_848d_4627_bf66_f5e3bb796be9.slice/crio-721358454f186acc16306cf2b65c416ef78912116111b2832c11fffc2c3eed7d WatchSource:0}: Error finding container 721358454f186acc16306cf2b65c416ef78912116111b2832c11fffc2c3eed7d: Status 404 returned error can't find the container with id 721358454f186acc16306cf2b65c416ef78912116111b2832c11fffc2c3eed7d
Jul 19 14:48:45.561380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.561340    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb766b755-6bk77" event={"ID":"d17ad151-848d-4627-bf66-f5e3bb796be9","Type":"ContainerStarted","Data":"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"}
Jul 19 14:48:45.561380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.561388    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb766b755-6bk77" event={"ID":"d17ad151-848d-4627-bf66-f5e3bb796be9","Type":"ContainerStarted","Data":"721358454f186acc16306cf2b65c416ef78912116111b2832c11fffc2c3eed7d"}
Jul 19 14:48:45.578409 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.578351    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cb766b755-6bk77" podStartSLOduration=1.287847927 podStartE2EDuration="1.578335659s" totalImagesPullingTime="290.487732ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:45.57591029 +0000 UTC m=+728.002387303" watchObservedRunningTime="2026-07-19 14:48:45.578335659 +0000 UTC m=+728.004812729"
Jul 19 14:48:45.607038 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.606998    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:45.607225 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.607201    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-grcdw" podUID="7caf3e27-e37f-400e-b947-faf7874cad0a" containerName="authorino" containerID="cri-o://cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c" gracePeriod=30
Jul 19 14:48:45.939336 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.939309    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:45.946828 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.946775    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h27vm\" (UniqueName: \"kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm\") pod \"7caf3e27-e37f-400e-b947-faf7874cad0a\" (UID: \"7caf3e27-e37f-400e-b947-faf7874cad0a\") "
Jul 19 14:48:45.949037 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:45.949003    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm" (OuterVolumeSpecName: "kube-api-access-h27vm") pod "7caf3e27-e37f-400e-b947-faf7874cad0a" (UID: "7caf3e27-e37f-400e-b947-faf7874cad0a"). InnerVolumeSpecName "kube-api-access-h27vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:48:46.047738 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.047699    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h27vm\" (UniqueName: \"kubernetes.io/projected/7caf3e27-e37f-400e-b947-faf7874cad0a-kube-api-access-h27vm\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:48:46.567846 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.567809    2585 generic.go:356] "Generic (PLEG): container finished" podID="7caf3e27-e37f-400e-b947-faf7874cad0a" containerID="cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c" exitCode=0
Jul 19 14:48:46.568020 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.567916    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-grcdw"
Jul 19 14:48:46.568020 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.567951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-grcdw" event={"ID":"7caf3e27-e37f-400e-b947-faf7874cad0a","Type":"ContainerDied","Data":"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"}
Jul 19 14:48:46.568020 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.568004    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-grcdw" event={"ID":"7caf3e27-e37f-400e-b947-faf7874cad0a","Type":"ContainerDied","Data":"9e5490178c063f6f327de6d4922993f826f4110c5bc1678c34ab472c230cece8"}
Jul 19 14:48:46.568219 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.568027    2585 scope.go:117] "RemoveContainer" containerID="cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"
Jul 19 14:48:46.576852 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.576830    2585 scope.go:117] "RemoveContainer" containerID="cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"
Jul 19 14:48:46.577168 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:48:46.577147    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c\": container with ID starting with cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c not found: ID does not exist" containerID="cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"
Jul 19 14:48:46.577240 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.577178    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c"} err="failed to get container status \"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c\": rpc error: code = NotFound desc = could not find container \"cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c\": container with ID starting with cd717fa8193a5675dac1fdaf314828021bd21d5e7688b83a64c74304c7a7f92c not found: ID does not exist"
Jul 19 14:48:46.582472 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.582445    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:46.584149 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:46.584127    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-grcdw"]
Jul 19 14:48:48.422188 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:48.422158    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7caf3e27-e37f-400e-b947-faf7874cad0a" path="/var/lib/kubelet/pods/7caf3e27-e37f-400e-b947-faf7874cad0a/volumes"
Jul 19 14:48:50.410322 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:50.410281    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.500955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:50.500913    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.557537 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:50.557500    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:50.779274 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:50.779236    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:53.576967 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:53.576918    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:48:53.674834 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:48:53.674768    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:02.560415 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:02.560382    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:02.608663 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:02.608610    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:24.076020 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.075979    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:24.173281 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.173248    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:24.576928 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.576889    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"]
Jul 19 14:49:24.577333 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.577319    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7caf3e27-e37f-400e-b947-faf7874cad0a" containerName="authorino"
Jul 19 14:49:24.577391 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.577336    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7caf3e27-e37f-400e-b947-faf7874cad0a" containerName="authorino"
Jul 19 14:49:24.577445 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.577432    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7caf3e27-e37f-400e-b947-faf7874cad0a" containerName="authorino"
Jul 19 14:49:24.579611 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.579581    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:24.599763 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.599725    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"]
Jul 19 14:49:24.599936 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.599831    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.603644 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.603618    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 19 14:49:24.603806 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.603701    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b4zp5\""
Jul 19 14:49:24.693184 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.693143    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deb97b98-9639-492c-b3ef-a55cb48bd128-openshift-service-ca\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.693380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.693221    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deb97b98-9639-492c-b3ef-a55cb48bd128-maas-api-tls\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.693380 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.693239    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpt5t\" (UniqueName: \"kubernetes.io/projected/deb97b98-9639-492c-b3ef-a55cb48bd128-kube-api-access-qpt5t\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.793766 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.793723    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deb97b98-9639-492c-b3ef-a55cb48bd128-maas-api-tls\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.793766 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.793762    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpt5t\" (UniqueName: \"kubernetes.io/projected/deb97b98-9639-492c-b3ef-a55cb48bd128-kube-api-access-qpt5t\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.794035 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.793837    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deb97b98-9639-492c-b3ef-a55cb48bd128-openshift-service-ca\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.794437 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.794418    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deb97b98-9639-492c-b3ef-a55cb48bd128-openshift-service-ca\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.796246 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.796225    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deb97b98-9639-492c-b3ef-a55cb48bd128-maas-api-tls\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.797464 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.797436    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:24.804080 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.804060    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpt5t\" (UniqueName: \"kubernetes.io/projected/deb97b98-9639-492c-b3ef-a55cb48bd128-kube-api-access-qpt5t\") pod \"maas-api-765d74d4c4-mg6lz\" (UID: \"deb97b98-9639-492c-b3ef-a55cb48bd128\") " pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:24.911886 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:24.911752    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:25.057061 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.057019    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"]
Jul 19 14:49:25.060722 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:49:25.060065    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb97b98_9639_492c_b3ef_a55cb48bd128.slice/crio-7cd343e2897f26817686fc725b6371ed5cec2e28bb01ad6e73978faea00f25e7 WatchSource:0}: Error finding container 7cd343e2897f26817686fc725b6371ed5cec2e28bb01ad6e73978faea00f25e7: Status 404 returned error can't find the container with id 7cd343e2897f26817686fc725b6371ed5cec2e28bb01ad6e73978faea00f25e7
Jul 19 14:49:25.116520 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.116490    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:25.207301 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.207216    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:25.208026 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.208006    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"]
Jul 19 14:49:25.212968 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.212947    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.215171 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.215146    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 19 14:49:25.215296 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.215214    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gn4ll\""
Jul 19 14:49:25.226881 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.226850    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"]
Jul 19 14:49:25.233458 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.233432    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cfn7l"]
Jul 19 14:49:25.237738 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.237714    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.246063 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.246037    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cfn7l"]
Jul 19 14:49:25.300372 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.300338    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25558de6-bb27-4ff5-816e-a47f98d9e12d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.300537 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.300377    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57zx9\" (UniqueName: \"kubernetes.io/projected/25558de6-bb27-4ff5-816e-a47f98d9e12d-kube-api-access-57zx9\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.401875 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.401841    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25558de6-bb27-4ff5-816e-a47f98d9e12d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.402071 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.401896    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm4c9\" (UniqueName: \"kubernetes.io/projected/1e0dc04d-83bd-4da1-8847-9602efff040b-kube-api-access-fm4c9\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.402071 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.402004    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57zx9\" (UniqueName: \"kubernetes.io/projected/25558de6-bb27-4ff5-816e-a47f98d9e12d-kube-api-access-57zx9\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.402203 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.402144    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e0dc04d-83bd-4da1-8847-9602efff040b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.402708 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.402688    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25558de6-bb27-4ff5-816e-a47f98d9e12d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.410118 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.410092    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57zx9\" (UniqueName: \"kubernetes.io/projected/25558de6-bb27-4ff5-816e-a47f98d9e12d-kube-api-access-57zx9\") pod \"payload-pre-processing-7677b774bb-v9dtg\" (UID: \"25558de6-bb27-4ff5-816e-a47f98d9e12d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.503731 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.503684    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e0dc04d-83bd-4da1-8847-9602efff040b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.504129 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.504101    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm4c9\" (UniqueName: \"kubernetes.io/projected/1e0dc04d-83bd-4da1-8847-9602efff040b-kube-api-access-fm4c9\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.504697 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.504659    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e0dc04d-83bd-4da1-8847-9602efff040b-plugins-config-volume\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.516537 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.516503    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm4c9\" (UniqueName: \"kubernetes.io/projected/1e0dc04d-83bd-4da1-8847-9602efff040b-kube-api-access-fm4c9\") pod \"payload-processing-5bf47cd988-cfn7l\" (UID: \"1e0dc04d-83bd-4da1-8847-9602efff040b\") " pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.524470 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.524444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:25.551704 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.551671    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:25.682512 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.682300    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"]
Jul 19 14:49:25.685427 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:49:25.685396    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25558de6_bb27_4ff5_816e_a47f98d9e12d.slice/crio-e780f3d51a5487588efe27c507e8fd0e155d928612d627a6370558c801c4460a WatchSource:0}: Error finding container e780f3d51a5487588efe27c507e8fd0e155d928612d627a6370558c801c4460a: Status 404 returned error can't find the container with id e780f3d51a5487588efe27c507e8fd0e155d928612d627a6370558c801c4460a
Jul 19 14:49:25.719461 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.719367    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-cfn7l"]
Jul 19 14:49:25.722246 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:49:25.722210    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0dc04d_83bd_4da1_8847_9602efff040b.slice/crio-dbfe97801c3fa0852ad2bc32ea0e7a376fb347cc8e00b06895fe07e1f507464a WatchSource:0}: Error finding container dbfe97801c3fa0852ad2bc32ea0e7a376fb347cc8e00b06895fe07e1f507464a: Status 404 returned error can't find the container with id dbfe97801c3fa0852ad2bc32ea0e7a376fb347cc8e00b06895fe07e1f507464a
Jul 19 14:49:25.765619 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.765471    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz" event={"ID":"deb97b98-9639-492c-b3ef-a55cb48bd128","Type":"ContainerStarted","Data":"7cd343e2897f26817686fc725b6371ed5cec2e28bb01ad6e73978faea00f25e7"}
Jul 19 14:49:25.768567 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.768465    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg" event={"ID":"25558de6-bb27-4ff5-816e-a47f98d9e12d","Type":"ContainerStarted","Data":"e780f3d51a5487588efe27c507e8fd0e155d928612d627a6370558c801c4460a"}
Jul 19 14:49:25.771694 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:25.771619    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l" event={"ID":"1e0dc04d-83bd-4da1-8847-9602efff040b","Type":"ContainerStarted","Data":"dbfe97801c3fa0852ad2bc32ea0e7a376fb347cc8e00b06895fe07e1f507464a"}
Jul 19 14:49:30.803048 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.803009    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l" event={"ID":"1e0dc04d-83bd-4da1-8847-9602efff040b","Type":"ContainerStarted","Data":"20c82fffb045a725ebaa52dd50a16882f0a50cfb664a1e0b68bc1d1e84c01ece"}
Jul 19 14:49:30.803549 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.803113    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:30.804725 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.804696    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz" event={"ID":"deb97b98-9639-492c-b3ef-a55cb48bd128","Type":"ContainerStarted","Data":"90ad09e9a1fc46e90aa852caf1da9d9d365379ef8fe9dc593ce5a03b4884cc16"}
Jul 19 14:49:30.804890 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.804807    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:30.806271 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.806245    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg" event={"ID":"25558de6-bb27-4ff5-816e-a47f98d9e12d","Type":"ContainerStarted","Data":"76f9727fda1162b961d93cea5813ef119cc2657c4b458752a1d59fe95ed939ac"}
Jul 19 14:49:30.806383 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.806304    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:30.820344 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.820290    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l" podStartSLOduration=1.520405955 podStartE2EDuration="5.820274722s" totalImagesPullingTime="4.299868767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:49:30.818216006 +0000 UTC m=+773.244693022" watchObservedRunningTime="2026-07-19 14:49:30.820274722 +0000 UTC m=+773.246751731"
Jul 19 14:49:30.834092 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.834027    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz" podStartSLOduration=1.8709586599999999 podStartE2EDuration="6.834011213s" totalImagesPullingTime="4.963052553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:49:30.833336144 +0000 UTC m=+773.259813159" watchObservedRunningTime="2026-07-19 14:49:30.834011213 +0000 UTC m=+773.260488226"
Jul 19 14:49:30.849745 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:30.849682    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg" podStartSLOduration=1.506197261 podStartE2EDuration="5.849653978s" totalImagesPullingTime="4.343456717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:49:30.848604358 +0000 UTC m=+773.275081374" watchObservedRunningTime="2026-07-19 14:49:30.849653978 +0000 UTC m=+773.276130991"
Jul 19 14:49:33.827748 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.827712    2585 generic.go:356] "Generic (PLEG): container finished" podID="54f417ba-7683-404b-add7-d30b4d19f264" containerID="e889b9a24e563f5778bac998b9f7b9271f9037642ca10c62b1119c623e6e4523" exitCode=0
Jul 19 14:49:33.828254 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.827775    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" event={"ID":"54f417ba-7683-404b-add7-d30b4d19f264","Type":"ContainerDied","Data":"e889b9a24e563f5778bac998b9f7b9271f9037642ca10c62b1119c623e6e4523"}
Jul 19 14:49:33.828254 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.828204    2585 scope.go:117] "RemoveContainer" containerID="e889b9a24e563f5778bac998b9f7b9271f9037642ca10c62b1119c623e6e4523"
Jul 19 14:49:33.829232 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.829195    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:49:33.829348 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.829285    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:49:33.829348 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:33.829337    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 14:49:34.834711 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:34.834676    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" event={"ID":"54f417ba-7683-404b-add7-d30b4d19f264","Type":"ContainerStarted","Data":"fc91e02ff350a5a7aa86f6adc340b9beb0fae8c988dcd1aaab073a89d1b89e55"}
Jul 19 14:49:34.912498 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:34.912463    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:49:34.913082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:34.913058    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lrqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 19 14:49:34.913140 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:34.913108    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" podUID="54f417ba-7683-404b-add7-d30b4d19f264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 19 14:49:35.912264 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:35.912230    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lrqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 19 14:49:35.912699 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:35.912302    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" podUID="54f417ba-7683-404b-add7-d30b4d19f264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 19 14:49:36.819890 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:36.819861    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-765d74d4c4-mg6lz"
Jul 19 14:49:36.912444 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:36.912410    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lrqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 19 14:49:36.912852 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:36.912493    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" podUID="54f417ba-7683-404b-add7-d30b4d19f264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 19 14:49:37.852340 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:37.852294    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:37.912832 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:37.912795    2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lrqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 19 14:49:37.913232 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:37.912859    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5" podUID="54f417ba-7683-404b-add7-d30b4d19f264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 19 14:49:37.947955 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:37.947918    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:38.916860 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:38.916829    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:49:38.917323 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:38.917074    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:49:38.917976 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:38.917954    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lrqb5"
Jul 19 14:49:41.816740 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:41.816711    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-cfn7l"
Jul 19 14:49:41.817196 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:41.816884    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-v9dtg"
Jul 19 14:49:43.886446 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.886380    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"]
Jul 19 14:49:43.895082 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.895057    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.902850 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.902811    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"]
Jul 19 14:49:43.903021 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.902817    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 19 14:49:43.903391 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.903364    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 19 14:49:43.903503 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.903453    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 19 14:49:43.904226 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.904202    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kzk9c\""
Jul 19 14:49:43.907986 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.907962    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.908100 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.908005    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.908100 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.908022    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.908224 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.908118    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55wkd\" (UniqueName: \"kubernetes.io/projected/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-kube-api-access-55wkd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.908224 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.908153    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.908224 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.908187    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:43.923270 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:43.923223    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:44.008942 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.008908    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.008968    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.008993    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009059    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55wkd\" (UniqueName: \"kubernetes.io/projected/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-kube-api-access-55wkd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009097    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009154 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009134    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009421    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009488 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009450    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.009687 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.009664    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.011549 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.011524    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.011718 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.011699    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.016914 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.016893    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55wkd\" (UniqueName: \"kubernetes.io/projected/3a078fe4-e5b8-4116-ae34-1626cc8fe70e-kube-api-access-55wkd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl\" (UID: \"3a078fe4-e5b8-4116-ae34-1626cc8fe70e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.216031 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.215933    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:44.362496 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.362466    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"]
Jul 19 14:49:44.364630 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:49:44.364591    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a078fe4_e5b8_4116_ae34_1626cc8fe70e.slice/crio-6157163b538c6a955bf1cb447cc723e3442f7ed4853692f0ecf2cebb058554d7 WatchSource:0}: Error finding container 6157163b538c6a955bf1cb447cc723e3442f7ed4853692f0ecf2cebb058554d7: Status 404 returned error can't find the container with id 6157163b538c6a955bf1cb447cc723e3442f7ed4853692f0ecf2cebb058554d7
Jul 19 14:49:44.899146 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:44.898951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl" event={"ID":"3a078fe4-e5b8-4116-ae34-1626cc8fe70e","Type":"ContainerStarted","Data":"6157163b538c6a955bf1cb447cc723e3442f7ed4853692f0ecf2cebb058554d7"}
Jul 19 14:49:45.274018 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:45.273966    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:45.325475 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:45.325419    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:45.905719 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:45.905611    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl" event={"ID":"3a078fe4-e5b8-4116-ae34-1626cc8fe70e","Type":"ContainerStarted","Data":"20f52de07231ae72e72781a4929a8f5c7e75a8253178587a1615d8fc522ba222"}
Jul 19 14:49:45.922914 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:45.922856    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl" podStartSLOduration=1.7121438260000001 podStartE2EDuration="2.922837158s" totalImagesPullingTime="1.210693332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:49:45.92032816 +0000 UTC m=+788.346805165" watchObservedRunningTime="2026-07-19 14:49:45.922837158 +0000 UTC m=+788.349314170"
Jul 19 14:49:48.779810 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:48.779743    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:49:54.216765 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:54.216712    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:54.217317 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:54.216823    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:54.229399 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:54.229370    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:49:54.970926 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:49:54.970893    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl"
Jul 19 14:50:11.728196 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:11.728148    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:20.725372 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:20.725334    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:50.381029 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.380992    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"]
Jul 19 14:50:50.385724 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.385698    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.388674 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.388649    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 19 14:50:50.398744 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.398717    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"]
Jul 19 14:50:50.427249 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.427210    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:50:50.541268 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541233    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.541495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541294    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.541495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541406    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.541495 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541485    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxl8c\" (UniqueName: \"kubernetes.io/projected/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-kube-api-access-mxl8c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.541687 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541523    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.541687 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.541560    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642713 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642616    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642713 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642673    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642809    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxl8c\" (UniqueName: \"kubernetes.io/projected/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-kube-api-access-mxl8c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642864    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642898    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.642974 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.642958    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.643197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.643093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.643197 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.643169    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.643340 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.643322    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.645348 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.645314    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.645507 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.645489    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.652643 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.652618    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxl8c\" (UniqueName: \"kubernetes.io/projected/d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0-kube-api-access-mxl8c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-g2zdc\" (UID: \"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.702704 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.702657    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:50:50.846098 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:50.846074    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"]
Jul 19 14:50:50.848236 ip-10-0-141-105 kubenswrapper[2585]: W0719 14:50:50.848207    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ed9ab5_3b98_4cf2_8143_f34e66f5bac0.slice/crio-683e8c1df34ea268a303877b0fb8df44a1e73c257174c2a0e69762b4efee06bd WatchSource:0}: Error finding container 683e8c1df34ea268a303877b0fb8df44a1e73c257174c2a0e69762b4efee06bd: Status 404 returned error can't find the container with id 683e8c1df34ea268a303877b0fb8df44a1e73c257174c2a0e69762b4efee06bd
Jul 19 14:50:51.247524 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:51.247491    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc" event={"ID":"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0","Type":"ContainerStarted","Data":"d1bf3b03ce205158aecd07e3f47985d88d8b1aa01928fa6a6abaa9d200e78df4"}
Jul 19 14:50:51.247713 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:51.247543    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc" event={"ID":"d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0","Type":"ContainerStarted","Data":"683e8c1df34ea268a303877b0fb8df44a1e73c257174c2a0e69762b4efee06bd"}
Jul 19 14:50:51.270103 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:51.270000    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc" podStartSLOduration=1.115407664 podStartE2EDuration="1.269980053s" totalImagesPullingTime="154.572389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:50:51.268624877 +0000 UTC m=+853.695101895" watchObservedRunningTime="2026-07-19 14:50:51.269980053 +0000 UTC m=+853.696457069"
Jul 19 14:50:56.922481 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:50:56.922437    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:00.702802 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:00.702742    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:51:00.703227 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:00.702846    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:51:00.715946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:00.715911    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:51:01.316865 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:01.316828    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-g2zdc"
Jul 19 14:51:38.125750 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:38.125664    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:51:38.129243 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:38.129215    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:51:49.026984 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.026834    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:51:49.027586 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.027075    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cb766b755-6bk77" podUID="d17ad151-848d-4627-bf66-f5e3bb796be9" containerName="authorino" containerID="cri-o://936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216" gracePeriod=30
Jul 19 14:51:49.299908 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.299881    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:51:49.404156 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.404123    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert\") pod \"d17ad151-848d-4627-bf66-f5e3bb796be9\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") "
Jul 19 14:51:49.404345 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.404178    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj68c\" (UniqueName: \"kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c\") pod \"d17ad151-848d-4627-bf66-f5e3bb796be9\" (UID: \"d17ad151-848d-4627-bf66-f5e3bb796be9\") "
Jul 19 14:51:49.406276 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.406240    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c" (OuterVolumeSpecName: "kube-api-access-sj68c") pod "d17ad151-848d-4627-bf66-f5e3bb796be9" (UID: "d17ad151-848d-4627-bf66-f5e3bb796be9"). InnerVolumeSpecName "kube-api-access-sj68c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:51:49.415274 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.415240    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d17ad151-848d-4627-bf66-f5e3bb796be9" (UID: "d17ad151-848d-4627-bf66-f5e3bb796be9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:51:49.505560 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.505517    2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d17ad151-848d-4627-bf66-f5e3bb796be9-tls-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:51:49.505560 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.505552    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj68c\" (UniqueName: \"kubernetes.io/projected/d17ad151-848d-4627-bf66-f5e3bb796be9-kube-api-access-sj68c\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:51:49.554499 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.554398    2585 generic.go:356] "Generic (PLEG): container finished" podID="d17ad151-848d-4627-bf66-f5e3bb796be9" containerID="936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216" exitCode=0
Jul 19 14:51:49.554499 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.554440    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb766b755-6bk77"
Jul 19 14:51:49.554499 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.554480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb766b755-6bk77" event={"ID":"d17ad151-848d-4627-bf66-f5e3bb796be9","Type":"ContainerDied","Data":"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"}
Jul 19 14:51:49.554747 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.554530    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb766b755-6bk77" event={"ID":"d17ad151-848d-4627-bf66-f5e3bb796be9","Type":"ContainerDied","Data":"721358454f186acc16306cf2b65c416ef78912116111b2832c11fffc2c3eed7d"}
Jul 19 14:51:49.554747 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.554548    2585 scope.go:117] "RemoveContainer" containerID="936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"
Jul 19 14:51:49.565792 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.565758    2585 scope.go:117] "RemoveContainer" containerID="936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"
Jul 19 14:51:49.566131 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:51:49.566111    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216\": container with ID starting with 936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216 not found: ID does not exist" containerID="936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"
Jul 19 14:51:49.566201 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.566142    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216"} err="failed to get container status \"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216\": rpc error: code = NotFound desc = could not find container \"936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216\": container with ID starting with 936218cd0eac61bf15bea266dad274c2931105f2365f350f095a7c1a17a46216 not found: ID does not exist"
Jul 19 14:51:49.567163 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.567142    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:49.578759 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.578730    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:51:49.589336 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:49.589311    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cb766b755-6bk77"]
Jul 19 14:51:50.423192 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:50.423152    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17ad151-848d-4627-bf66-f5e3bb796be9" path="/var/lib/kubelet/pods/d17ad151-848d-4627-bf66-f5e3bb796be9/volumes"
Jul 19 14:51:50.718299 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:50.718220    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:51:50.818944 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:51:50.818904    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:22.317526 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:52:22.317487    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:27.334750 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:52:27.334714    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:28.110362 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:52:28.110328    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:36.974757 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:52:36.974709    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:52:37.906329 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:52:37.906293    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:15.577946 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:15.577905    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:16.786359 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:16.786320    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:42.376652 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:42.376604    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:50.273335 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:50.273300    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:50.373307 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:50.373268    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:50.500381 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:50.500343    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:56.197377 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:56.197339    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:56.307177 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:56.307140    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:56.400633 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:56.400593    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:54:56.564332 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:54:56.564295    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:02.397128 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:55:02.397075    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:02.497030 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:55:02.496994    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:55:45.241804 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:55:45.241729    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:56:38.159265 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:56:38.159235    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:56:38.169149 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:56:38.169119    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 14:58:53.681525 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:58:53.681488    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:58:54.632885 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:58:54.632847    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 14:59:01.106320 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:01.106232    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:59:01.106726 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:01.106444    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" podUID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" containerName="manager" containerID="cri-o://6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781" gracePeriod=10
Jul 19 14:59:05.672447 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.672416    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:59:05.808727 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.808625    2585 generic.go:356] "Generic (PLEG): container finished" podID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" containerID="6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781" exitCode=0
Jul 19 14:59:05.808727 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.808675    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"
Jul 19 14:59:05.808727 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.808713    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" event={"ID":"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2","Type":"ContainerDied","Data":"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"}
Jul 19 14:59:05.809076 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.808769    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz" event={"ID":"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2","Type":"ContainerDied","Data":"e0068fdb07921b22ddb44fe77d22336124a091808143c7f5bb06a299a523dfe2"}
Jul 19 14:59:05.809076 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.808807    2585 scope.go:117] "RemoveContainer" containerID="6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"
Jul 19 14:59:05.815759 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.815738    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume\") pod \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") "
Jul 19 14:59:05.815899 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.815882    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgct6\" (UniqueName: \"kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6\") pod \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\" (UID: \"8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2\") "
Jul 19 14:59:05.816155 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.816133    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" (UID: "8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:59:05.816240 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.816227    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-extensions-socket-volume\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:59:05.817831 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.817806    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6" (OuterVolumeSpecName: "kube-api-access-lgct6") pod "8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" (UID: "8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2"). InnerVolumeSpecName "kube-api-access-lgct6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:59:05.818593 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.818575    2585 scope.go:117] "RemoveContainer" containerID="6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"
Jul 19 14:59:05.818883 ip-10-0-141-105 kubenswrapper[2585]: E0719 14:59:05.818862    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781\": container with ID starting with 6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781 not found: ID does not exist" containerID="6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"
Jul 19 14:59:05.818959 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.818896    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781"} err="failed to get container status \"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781\": rpc error: code = NotFound desc = could not find container \"6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781\": container with ID starting with 6970f319fe86e8120f83e0a006ae10ae99e9e16655e64ed6f5f1974ce7d86781 not found: ID does not exist"
Jul 19 14:59:05.917323 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:05.917290    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgct6\" (UniqueName: \"kubernetes.io/projected/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2-kube-api-access-lgct6\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 14:59:06.135111 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:06.135082    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:59:06.140951 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:06.140925    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-458lz"]
Jul 19 14:59:06.422995 ip-10-0-141-105 kubenswrapper[2585]: I0719 14:59:06.422915    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" path="/var/lib/kubelet/pods/8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2/volumes"
Jul 19 15:00:00.144931 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.144894    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"]
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145362    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" containerName="manager"
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145374    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" containerName="manager"
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145385    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17ad151-848d-4627-bf66-f5e3bb796be9" containerName="authorino"
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145390    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17ad151-848d-4627-bf66-f5e3bb796be9" containerName="authorino"
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145454    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17ad151-848d-4627-bf66-f5e3bb796be9" containerName="authorino"
Jul 19 15:00:00.147419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.145465    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c73b458-2fc2-40b0-8e7c-1f2c9bef98d2" containerName="manager"
Jul 19 15:00:00.148459 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.148444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"
Jul 19 15:00:00.150952 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.150933    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:00:00.155936 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.155897    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"]
Jul 19 15:00:00.160343 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.160320    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"
Jul 19 15:00:00.301532 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.301505    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"]
Jul 19 15:00:00.303701 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:00:00.303665    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce6ed70_fa9b_4372_b0d3_248d8cac5a07.slice/crio-4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c WatchSource:0}: Error finding container 4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c: Status 404 returned error can't find the container with id 4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c
Jul 19 15:00:00.305659 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:00.305642    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:00:01.087476 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:01.087440    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44" event={"ID":"cce6ed70-fa9b-4372-b0d3-248d8cac5a07","Type":"ContainerStarted","Data":"4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c"}
Jul 19 15:00:02.314664 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.314624    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"]
Jul 19 15:00:02.319768 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.319741    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.322129 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.322106    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vgftl\""
Jul 19 15:00:02.329840 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.329815    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"]
Jul 19 15:00:02.418375 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.418338    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llx57\" (UniqueName: \"kubernetes.io/projected/bfec1624-e981-43b1-bad4-6c043daf08b6-kube-api-access-llx57\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.418579 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.418386    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfec1624-e981-43b1-bad4-6c043daf08b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.519721 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.519677    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llx57\" (UniqueName: \"kubernetes.io/projected/bfec1624-e981-43b1-bad4-6c043daf08b6-kube-api-access-llx57\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.519941 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.519737    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfec1624-e981-43b1-bad4-6c043daf08b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.520327 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.520297    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfec1624-e981-43b1-bad4-6c043daf08b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.528010 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.527980    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llx57\" (UniqueName: \"kubernetes.io/projected/bfec1624-e981-43b1-bad4-6c043daf08b6-kube-api-access-llx57\") pod \"kuadrant-operator-controller-manager-86878c769f-bknhk\" (UID: \"bfec1624-e981-43b1-bad4-6c043daf08b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.631060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.630968    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:02.774387 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:02.774246    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"]
Jul 19 15:00:02.776974 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:00:02.776947    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfec1624_e981_43b1_bad4_6c043daf08b6.slice/crio-9bdda99310da876431bb524ac6f01502886155e9bd70031346ddb0573cd30254 WatchSource:0}: Error finding container 9bdda99310da876431bb524ac6f01502886155e9bd70031346ddb0573cd30254: Status 404 returned error can't find the container with id 9bdda99310da876431bb524ac6f01502886155e9bd70031346ddb0573cd30254
Jul 19 15:00:03.100964 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:03.100924    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk" event={"ID":"bfec1624-e981-43b1-bad4-6c043daf08b6","Type":"ContainerStarted","Data":"e825fe52211ebba6e251f12b4b83625571000ce80d0437934e5dda9ce3b61649"}
Jul 19 15:00:03.101365 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:03.100991    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk" event={"ID":"bfec1624-e981-43b1-bad4-6c043daf08b6","Type":"ContainerStarted","Data":"9bdda99310da876431bb524ac6f01502886155e9bd70031346ddb0573cd30254"}
Jul 19 15:00:03.101365 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:03.101090    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:03.122286 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:03.122221    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk" podStartSLOduration=1.122197333 podStartE2EDuration="1.122197333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:00:03.118907991 +0000 UTC m=+1405.545385011" watchObservedRunningTime="2026-07-19 15:00:03.122197333 +0000 UTC m=+1405.548674347"
Jul 19 15:00:05.113091 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:05.113055    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44" event={"ID":"cce6ed70-fa9b-4372-b0d3-248d8cac5a07","Type":"ContainerStarted","Data":"4b12c611677a9bdeb429f6e903d0d47fcf089edea6d181319db6c0bdb5572247"}
Jul 19 15:00:05.129188 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:05.129132    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44" podStartSLOduration=1.079196712 podStartE2EDuration="5.129113947s" totalImagesPullingTime="4.049917235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:00:05.126525171 +0000 UTC m=+1407.553002184" watchObservedRunningTime="2026-07-19 15:00:05.129113947 +0000 UTC m=+1407.555590959"
Jul 19 15:00:06.119485 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:06.119439    2585 generic.go:356] "Generic (PLEG): container finished" podID="cce6ed70-fa9b-4372-b0d3-248d8cac5a07" containerID="4b12c611677a9bdeb429f6e903d0d47fcf089edea6d181319db6c0bdb5572247" exitCode=0
Jul 19 15:00:06.119875 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:06.119567    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44" event={"ID":"cce6ed70-fa9b-4372-b0d3-248d8cac5a07","Type":"ContainerDied","Data":"4b12c611677a9bdeb429f6e903d0d47fcf089edea6d181319db6c0bdb5572247"}
Jul 19 15:00:06.781556 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:06.781519    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:00:07.267401 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:07.267377    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"
Jul 19 15:00:08.130800 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:08.130763    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44"
Jul 19 15:00:08.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:08.130830    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741220-5fp44" event={"ID":"cce6ed70-fa9b-4372-b0d3-248d8cac5a07","Type":"ContainerDied","Data":"4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c"}
Jul 19 15:00:08.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:08.130886    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e5cfb74cfa621f70ad4a0cf64294bbd435a70e31466d0cc395abd31a3c44e8c"
Jul 19 15:00:14.109522 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:14.109490    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bknhk"
Jul 19 15:00:41.251031 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:00:41.250929    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:01:22.416119 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:01:22.416082    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:01:38.195223 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:01:38.195190    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:01:38.203941 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:01:38.203918    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:02:41.018458 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:02:41.018418    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:05.297520 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.297469    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:05.298197 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.298176    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce6ed70-fa9b-4372-b0d3-248d8cac5a07" containerName="cleanup"
Jul 19 15:03:05.298293 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.298199    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce6ed70-fa9b-4372-b0d3-248d8cac5a07" containerName="cleanup"
Jul 19 15:03:05.298293 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.298288    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce6ed70-fa9b-4372-b0d3-248d8cac5a07" containerName="cleanup"
Jul 19 15:03:05.301623 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.301597    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.303984 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.303962    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-89c2edb3-kserve-self-signed-certs\""
Jul 19 15:03:05.309275 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.309194    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:05.321151 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.321115    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:05.443494 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.443699 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443557    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.443699 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443595    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.443864 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443698    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.443864 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443743    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5rvf\" (UniqueName: \"kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.443864 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.443774    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544332 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544294    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544540 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544362    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544540 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544389    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544540 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544431    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544540 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544447    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5rvf\" (UniqueName: \"kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544769 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544929 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544858    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.544929 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.544867    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.545106 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.545091    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.547087 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.547068    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.547202 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.547097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.553202 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.553141    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5rvf\" (UniqueName: \"kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf\") pod \"e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") " pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.612077 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.612033    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:05.759062 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:05.759029    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:05.760862 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:03:05.760828    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde7c769_0c01_4bd2_a4be_c053d6f0dddf.slice/crio-3cc88a7fd0442b7d5dfaf67ba77279e7c129b04a1f1858fd9a6c410890570648 WatchSource:0}: Error finding container 3cc88a7fd0442b7d5dfaf67ba77279e7c129b04a1f1858fd9a6c410890570648: Status 404 returned error can't find the container with id 3cc88a7fd0442b7d5dfaf67ba77279e7c129b04a1f1858fd9a6c410890570648
Jul 19 15:03:06.076284 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:06.076192    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" event={"ID":"fde7c769-0c01-4bd2-a4be-c053d6f0dddf","Type":"ContainerStarted","Data":"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"}
Jul 19 15:03:06.076284 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:06.076233    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" event={"ID":"fde7c769-0c01-4bd2-a4be-c053d6f0dddf","Type":"ContainerStarted","Data":"3cc88a7fd0442b7d5dfaf67ba77279e7c129b04a1f1858fd9a6c410890570648"}
Jul 19 15:03:06.093096 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:06.093040    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" podStartSLOduration=1.093023699 podStartE2EDuration="1.093023699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:03:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:03:06.090576634 +0000 UTC m=+1588.517053646" watchObservedRunningTime="2026-07-19 15:03:06.093023699 +0000 UTC m=+1588.519500711"
Jul 19 15:03:15.613156 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:15.613113    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:15.613156 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:15.613162    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:15.626284 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:15.626250    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:16.145154 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:16.145115    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:27.938604 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:27.938566    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:29.185815 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:29.185762    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:29.205500 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:29.205453    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:03:44.208627 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.208502    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" podUID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" containerName="main" containerID="cri-o://5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319" gracePeriod=45
Jul 19 15:03:44.470369 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.470288    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:44.614112 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614069    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614128    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614193    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614247    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614273    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5rvf\" (UniqueName: \"kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614297    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs\") pod \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\" (UID: \"fde7c769-0c01-4bd2-a4be-c053d6f0dddf\") "
Jul 19 15:03:44.614587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614380    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:03:44.614587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614415    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache" (OuterVolumeSpecName: "model-cache") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:03:44.614587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614572    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home" (OuterVolumeSpecName: "home") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:03:44.614715 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614584    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tmp-dir\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:44.614715 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.614604    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-model-cache\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:44.616629 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.616598    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm" (OuterVolumeSpecName: "dshm") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:03:44.616629 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.616620    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:03:44.616818 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.616678    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf" (OuterVolumeSpecName: "kube-api-access-j5rvf") pod "fde7c769-0c01-4bd2-a4be-c053d6f0dddf" (UID: "fde7c769-0c01-4bd2-a4be-c053d6f0dddf"). InnerVolumeSpecName "kube-api-access-j5rvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:03:44.716102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.716060    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-home\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:44.716102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.716095    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5rvf\" (UniqueName: \"kubernetes.io/projected/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-kube-api-access-j5rvf\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:44.716102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.716106    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-tls-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:44.716102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:44.716115    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde7c769-0c01-4bd2-a4be-c053d6f0dddf-dshm\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:03:45.291344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.291305    2585 generic.go:356] "Generic (PLEG): container finished" podID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" containerID="5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319" exitCode=0
Jul 19 15:03:45.291741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.291401    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"
Jul 19 15:03:45.291741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.291438    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" event={"ID":"fde7c769-0c01-4bd2-a4be-c053d6f0dddf","Type":"ContainerDied","Data":"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"}
Jul 19 15:03:45.291741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.291486    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9" event={"ID":"fde7c769-0c01-4bd2-a4be-c053d6f0dddf","Type":"ContainerDied","Data":"3cc88a7fd0442b7d5dfaf67ba77279e7c129b04a1f1858fd9a6c410890570648"}
Jul 19 15:03:45.291741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.291509    2585 scope.go:117] "RemoveContainer" containerID="5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"
Jul 19 15:03:45.302362 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.302337    2585 scope.go:117] "RemoveContainer" containerID="5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"
Jul 19 15:03:45.302732 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:03:45.302701    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319\": container with ID starting with 5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319 not found: ID does not exist" containerID="5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"
Jul 19 15:03:45.302861 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.302737    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319"} err="failed to get container status \"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319\": rpc error: code = NotFound desc = could not find container \"5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319\": container with ID starting with 5fb359a7d29e5f601cf786ae06d01297b8af088dac578305e873ef04fca4d319 not found: ID does not exist"
Jul 19 15:03:45.312460 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.312427    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:45.315013 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:45.314987    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-89c2edb3-kserve-5c7bd944c9-dgvm9"]
Jul 19 15:03:46.422522 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:03:46.422488    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" path="/var/lib/kubelet/pods/fde7c769-0c01-4bd2-a4be-c053d6f0dddf/volumes"
Jul 19 15:05:26.116673 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:05:26.116637    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:38.229992 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:38.229964    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:06:38.238277 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:38.238253    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:06:43.240850 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:43.240814    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:43.357878 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:43.357845    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:43.731167 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:43.731135    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:49.250867 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.250817    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:49.351267 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.351229    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:49.480458 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.480425    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:49.770362 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.770324    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:06:49.770816 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.770773    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" containerName="main"
Jul 19 15:06:49.770816 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.770815    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" containerName="main"
Jul 19 15:06:49.771009 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.770914    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fde7c769-0c01-4bd2-a4be-c053d6f0dddf" containerName="main"
Jul 19 15:06:49.774134 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.774112    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.776516 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.776496    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-28590d9c-openshift-default-dockercfg-8xj4k\""
Jul 19 15:06:49.793043 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.793000    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:49.795044 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.795012    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:06:49.834184 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834138    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5cwc\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834364 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834193    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834364 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834221    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834364 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834473 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834394    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834473 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834437    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834473 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834460    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834567 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834514    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.834567 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.834537    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.935585 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.935818 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935629    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.935818 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935662    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.935818 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935679    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.935818 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935712    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935945    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5cwc\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.935992    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936018    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936233 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936165    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936233 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936188    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936376 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936354    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936493 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936473    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.936832 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.936809    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.938100 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.938083    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.938498 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.938481    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.958997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.958959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:49.959225 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:49.959188    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5cwc\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc\") pod \"e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") " pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:50.032638 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.032540    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:50.085760 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.085717    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:50.233340 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.233303    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:06:50.234408 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:06:50.234382    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd176bd2_54c3_470b_affb_aabd4d6d7776.slice/crio-72e573774f4ccfaf6e9e8b2793aa9cc0c105825bb936568ac84250d31c63cf46 WatchSource:0}: Error finding container 72e573774f4ccfaf6e9e8b2793aa9cc0c105825bb936568ac84250d31c63cf46: Status 404 returned error can't find the container with id 72e573774f4ccfaf6e9e8b2793aa9cc0c105825bb936568ac84250d31c63cf46
Jul 19 15:06:50.236610 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.236573    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:06:50.236704 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.236668    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:06:50.236704 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.236699    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:06:50.288310 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:50.288277    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerStarted","Data":"72e573774f4ccfaf6e9e8b2793aa9cc0c105825bb936568ac84250d31c63cf46"}
Jul 19 15:06:51.295555 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:51.295520    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerStarted","Data":"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"}
Jul 19 15:06:51.315829 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:51.315753    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podStartSLOduration=2.31573696 podStartE2EDuration="2.31573696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:06:51.315621479 +0000 UTC m=+1813.742098494" watchObservedRunningTime="2026-07-19 15:06:51.31573696 +0000 UTC m=+1813.742213983"
Jul 19 15:06:52.086577 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.086539    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:52.092271 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.092239    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:52.301043 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.301015    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:52.302133 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.302113    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:06:52.648230 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.648191    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:52.732828 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.732793    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:52.799432 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.799395    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:52.898747 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:52.898651    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:53.202775 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:53.202681    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:06:53.303497 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:06:53.303451    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:08.343014 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:08.342970    2585 patch_prober.go:28] interesting pod/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 19 15:07:08.345360 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:08.343102    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 19 15:07:09.400077 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.400041    2585 generic.go:356] "Generic (PLEG): container finished" podID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerID="168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051" exitCode=0
Jul 19 15:07:09.400488 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.400113    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerDied","Data":"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"}
Jul 19 15:07:09.400544 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.400529    2585 scope.go:117] "RemoveContainer" containerID="168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"
Jul 19 15:07:09.401661 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.401631    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:07:09.401739 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.401722    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:07:09.401821 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:09.401776    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:07:10.407059 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:10.407026    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerStarted","Data":"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"}
Jul 19 15:07:11.086386 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:11.086340    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:11.086601 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:11.086577    2585 patch_prober.go:28] interesting pod/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:07:11.086648 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:11.086631    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:07:12.086775 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:12.086735    2585 patch_prober.go:28] interesting pod/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:07:12.087228 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:12.086834    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:07:13.086209 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:13.086163    2585 patch_prober.go:28] interesting pod/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:07:13.086415 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:13.086233    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:07:14.086793 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:14.086755    2585 patch_prober.go:28] interesting pod/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 19 15:07:14.087290 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:14.086839    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 19 15:07:15.089766 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:15.089732    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:15.090214 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:15.090033    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:15.090730 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:15.090708    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:28.675112 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:28.675073    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:33.358259 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:33.358221    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:33.494060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:33.494025    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:07:33.494305 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:33.494277    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy" containerID="cri-o://b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a" gracePeriod=30
Jul 19 15:07:33.514975 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:33.514939    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:33.813207 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:33.813167    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:34.025595 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:34.025559    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:34.124046 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:34.123945    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:34.237908 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:34.237871    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:34.553043 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:34.552998    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:34.898213 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:34.898096    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:36.977937 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:36.977903    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:37.088239 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:37.088209    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:37.252690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:37.252603    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:37.570238 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:37.570138    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:37.671326 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:37.671292    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:07:38.766142 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.766076    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:38.932420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932380    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932430    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932641 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932450    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932641 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932483    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932641 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932562    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932641 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932618    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5cwc\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932670    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932696    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932728    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket\") pod \"cd176bd2-54c3-470b-affb-aabd4d6d7776\" (UID: \"cd176bd2-54c3-470b-affb-aabd4d6d7776\") "
Jul 19 15:07:38.932869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932743    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:07:38.932869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.932745    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:07:38.933134 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933095    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:07:38.933215 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933195    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd176bd2-54c3-470b-affb-aabd4d6d7776-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:38.933272 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933218    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:38.933272 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933232    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:38.933345 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933283    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:07:38.933478 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.933456    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:07:38.935073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.935039    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:07:38.935194 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.935108    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc" (OuterVolumeSpecName: "kube-api-access-k5cwc") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "kube-api-access-k5cwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:07:38.935334 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.935313    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:07:38.935373 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:38.935357    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd176bd2-54c3-470b-affb-aabd4d6d7776" (UID: "cd176bd2-54c3-470b-affb-aabd4d6d7776"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034129    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5cwc\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-kube-api-access-k5cwc\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034162    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034182    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034191    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034200    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.034231 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.034208    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd176bd2-54c3-470b-affb-aabd4d6d7776-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:07:39.565320 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.565283    2585 generic.go:356] "Generic (PLEG): container finished" podID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerID="b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a" exitCode=0
Jul 19 15:07:39.565513 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.565345    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"
Jul 19 15:07:39.565513 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.565368    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerDied","Data":"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"}
Jul 19 15:07:39.565513 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.565424    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k" event={"ID":"cd176bd2-54c3-470b-affb-aabd4d6d7776","Type":"ContainerDied","Data":"72e573774f4ccfaf6e9e8b2793aa9cc0c105825bb936568ac84250d31c63cf46"}
Jul 19 15:07:39.565513 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.565447    2585 scope.go:117] "RemoveContainer" containerID="b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"
Jul 19 15:07:39.577543 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.577516    2585 scope.go:117] "RemoveContainer" containerID="168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"
Jul 19 15:07:39.586837 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.586814    2585 scope.go:117] "RemoveContainer" containerID="b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"
Jul 19 15:07:39.587140 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:07:39.587118    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a\": container with ID starting with b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a not found: ID does not exist" containerID="b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"
Jul 19 15:07:39.587220 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.587156    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a"} err="failed to get container status \"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a\": rpc error: code = NotFound desc = could not find container \"b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a\": container with ID starting with b108e7b5d5618393294fb2962f057d88c7707e4e9593672025e9c356844d3d9a not found: ID does not exist"
Jul 19 15:07:39.587220 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.587182    2585 scope.go:117] "RemoveContainer" containerID="168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"
Jul 19 15:07:39.587481 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:07:39.587458    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051\": container with ID starting with 168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051 not found: ID does not exist" containerID="168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"
Jul 19 15:07:39.587524 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.587490    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051"} err="failed to get container status \"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051\": rpc error: code = NotFound desc = could not find container \"168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051\": container with ID starting with 168a24a1a0b095c8752aa7ae70d0173f724943c8a1c02cd5ffb4d0751bb0c051 not found: ID does not exist"
Jul 19 15:07:39.588432 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.588413    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:07:39.592002 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:39.591980    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-28590d9c-openshift-default-8444fc9575-skq2k"]
Jul 19 15:07:40.423051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:07:40.423011    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" path="/var/lib/kubelet/pods/cd176bd2-54c3-470b-affb-aabd4d6d7776/volumes"
Jul 19 15:08:02.025743 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.025693    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"]
Jul 19 15:08:02.026404 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.026382    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:02.026517 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.026443    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:02.026641 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.026627    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:02.026711 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.026647    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:02.030060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.030035    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:02.032466 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.032440    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:08:02.037102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.037063    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"]
Jul 19 15:08:02.142519 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.142462    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n\" (UID: \"bda6ac71-4daf-4249-9def-2406c99685da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:02.243962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.243918    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n\" (UID: \"bda6ac71-4daf-4249-9def-2406c99685da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:02.244628 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.244602    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n\" (UID: \"bda6ac71-4daf-4249-9def-2406c99685da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:02.342688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.342583    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:02.483562 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.483531    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"]
Jul 19 15:08:02.485473 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:08:02.485436    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda6ac71_4daf_4249_9def_2406c99685da.slice/crio-f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674 WatchSource:0}: Error finding container f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674: Status 404 returned error can't find the container with id f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674
Jul 19 15:08:02.684945 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:02.684910    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n" event={"ID":"bda6ac71-4daf-4249-9def-2406c99685da","Type":"ContainerStarted","Data":"f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674"}
Jul 19 15:08:03.692054 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:03.692014    2585 generic.go:356] "Generic (PLEG): container finished" podID="bda6ac71-4daf-4249-9def-2406c99685da" containerID="bda44bac40065c4e0bca624d997d31372253455750ad851169d2dea879841c31" exitCode=0
Jul 19 15:08:03.692458 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:03.692094    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n" event={"ID":"bda6ac71-4daf-4249-9def-2406c99685da","Type":"ContainerDied","Data":"bda44bac40065c4e0bca624d997d31372253455750ad851169d2dea879841c31"}
Jul 19 15:08:04.836360 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:04.836332    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:04.971859 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:04.971755    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca\") pod \"bda6ac71-4daf-4249-9def-2406c99685da\" (UID: \"bda6ac71-4daf-4249-9def-2406c99685da\") "
Jul 19 15:08:04.972202 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:04.972171    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bda6ac71-4daf-4249-9def-2406c99685da" (UID: "bda6ac71-4daf-4249-9def-2406c99685da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:08:05.073350 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:05.073306    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda6ac71-4daf-4249-9def-2406c99685da-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:08:05.704034 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:05.704001    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"
Jul 19 15:08:05.704289 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:05.704000    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n" event={"ID":"bda6ac71-4daf-4249-9def-2406c99685da","Type":"ContainerDied","Data":"f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674"}
Jul 19 15:08:05.704289 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:05.704126    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f99403e442d77f36e4c6da8375cef503874f47dcf6c14c281994e9ea89944674"
Jul 19 15:08:12.401548 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:12.401516    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:17.098752 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.098705    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:17.099227 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.099206    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"]
Jul 19 15:08:17.101046 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.101025    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c1c8ecb5-vd57n"]
Jul 19 15:08:17.267369 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.267335    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:17.495873 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.495839    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:17.728634 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:17.728601    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:18.023378 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023341    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:08:18.023825 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023810    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda6ac71-4daf-4249-9def-2406c99685da" containerName="revoke-keys"
Jul 19 15:08:18.023881 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023827    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda6ac71-4daf-4249-9def-2406c99685da" containerName="revoke-keys"
Jul 19 15:08:18.023881 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023838    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:18.023881 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023845    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd176bd2-54c3-470b-affb-aabd4d6d7776" containerName="istio-proxy"
Jul 19 15:08:18.023975 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.023935    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda6ac71-4daf-4249-9def-2406c99685da" containerName="revoke-keys"
Jul 19 15:08:18.027107 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.027085    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.029369 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.029348    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-79a97677-openshift-default-dockercfg-bq5mk\""
Jul 19 15:08:18.038811 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.038758    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:08:18.042140 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.042109    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:18.200959 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.200921    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.200959 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.200961    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jlk\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201103    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201155    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201171    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201186    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.201396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.201218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.302758 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jlk\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.302758 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302726    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.302758 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302756    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302771    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302847    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302873    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302894    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302915    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.302986    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303354 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.303232    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303514 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.303445    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303514 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.303487    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303698 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.303620    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.303698 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.303621    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.305247 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.305225    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.305517 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.305497    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.311154 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.311129    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.311426 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.311402    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jlk\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk\") pod \"e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") " pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.339176 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.339134    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:18.424657 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.424625    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda6ac71-4daf-4249-9def-2406c99685da" path="/var/lib/kubelet/pods/bda6ac71-4daf-4249-9def-2406c99685da/volumes"
Jul 19 15:08:18.490377 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.490348    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:08:18.491674 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:08:18.491647    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25d773c_a786_4fea_bd4a_c05e5afd49ad.slice/crio-47628a279e73af6fbc1a00ac015bc98ecb567db83ead186f749f276f23ffeb98 WatchSource:0}: Error finding container 47628a279e73af6fbc1a00ac015bc98ecb567db83ead186f749f276f23ffeb98: Status 404 returned error can't find the container with id 47628a279e73af6fbc1a00ac015bc98ecb567db83ead186f749f276f23ffeb98
Jul 19 15:08:18.493993 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.493955    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:18.494077 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.494052    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:18.494125 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.494100    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:18.675341 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.675298    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:18.777752 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.777717    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerStarted","Data":"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"}
Jul 19 15:08:18.777988 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.777767    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerStarted","Data":"47628a279e73af6fbc1a00ac015bc98ecb567db83ead186f749f276f23ffeb98"}
Jul 19 15:08:18.810670 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:18.810616    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podStartSLOduration=0.810599759 podStartE2EDuration="810.599759ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:08:18.809448175 +0000 UTC m=+1901.235925180" watchObservedRunningTime="2026-07-19 15:08:18.810599759 +0000 UTC m=+1901.237076772"
Jul 19 15:08:19.339382 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:19.339342    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:19.344480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:19.344442    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:19.782808 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:19.782765    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:19.783934 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:19.783916    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:20.227463 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:20.227426    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:20.327462 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:20.327428    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:20.444669 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:20.444632    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:20.970194 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:20.970147    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:21.071393 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:21.071360    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:08:34.784451 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.784358    2585 patch_prober.go:28] interesting pod/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 19 15:08:34.784957 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.784447    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 19 15:08:34.877663 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.877571    2585 generic.go:356] "Generic (PLEG): container finished" podID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerID="001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d" exitCode=0
Jul 19 15:08:34.877663 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.877613    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerDied","Data":"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"}
Jul 19 15:08:34.878060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.878044    2585 scope.go:117] "RemoveContainer" containerID="001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"
Jul 19 15:08:34.879123 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.879092    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:34.879193 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.879165    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:34.879229 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:34.879197    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:08:35.884812 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:35.884745    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerStarted","Data":"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"}
Jul 19 15:08:36.339425 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:36.339374    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:36.339831 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:36.339761    2585 patch_prober.go:28] interesting pod/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 19 15:08:36.339940 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:36.339860    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 19 15:08:37.339962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:37.339914    2585 patch_prober.go:28] interesting pod/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 19 15:08:37.340620 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:37.340005    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 19 15:08:38.340286 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:38.340239    2585 patch_prober.go:28] interesting pod/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 19 15:08:38.340807 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:38.340334    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 19 15:08:39.339809 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:39.339735    2585 patch_prober.go:28] interesting pod/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 19 15:08:39.340024 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:39.339858    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 19 15:08:40.343915 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:40.343878    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:40.344325 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:40.344149    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:40.344797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:40.344763    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:08:56.529493 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:08:56.529460    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:01.105939 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:01.105906    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:02.306764 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:02.306721    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:09:02.307185 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:02.307015    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy" containerID="cri-o://ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc" gracePeriod=30
Jul 19 15:09:02.323145 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:02.323105    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:02.565414 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:02.565330    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:02.790504 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:02.790466    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:03.092588 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:03.092545    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:03.608455 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:03.608422    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:05.277531 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:05.277452    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:05.374364 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:05.374333    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:05.444056 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:05.444022    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:05.669612 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:05.669526    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:05.955010 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:05.954918    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:06.053887 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:06.053850    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:07.587016 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.586942    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:09:07.602152 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602121    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602328 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602190    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602328 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602214    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602328 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602309    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602356    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2jlk\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602403    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602437    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602472    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602543    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:09:07.602688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602561    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert\") pod \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\" (UID: \"b25d773c-a786-4fea-bd4a-c05e5afd49ad\") "
Jul 19 15:09:07.602688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602571    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:09:07.602873 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602769    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:09:07.602930 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602912    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.602991 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602931    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.602991 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.602946    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.603108 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.603071    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:09:07.603167 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.603126    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:09:07.604704 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.604678    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:09:07.604970 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.604942    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:09:07.605076 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.604974    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk" (OuterVolumeSpecName: "kube-api-access-b2jlk") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "kube-api-access-b2jlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:09:07.605151 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.605074    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b25d773c-a786-4fea-bd4a-c05e5afd49ad" (UID: "b25d773c-a786-4fea-bd4a-c05e5afd49ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:09:07.703923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703878    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.703923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703912    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2jlk\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-kube-api-access-b2jlk\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.703923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703923    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.703923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703933    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.704204 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703941    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:07.704204 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:07.703949    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25d773c-a786-4fea-bd4a-c05e5afd49ad-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:08.059643 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.059597    2585 generic.go:356] "Generic (PLEG): container finished" podID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerID="ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc" exitCode=0
Jul 19 15:09:08.059643 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.059644    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"
Jul 19 15:09:08.059968 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.059681    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerDied","Data":"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"}
Jul 19 15:09:08.059968 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.059749    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8" event={"ID":"b25d773c-a786-4fea-bd4a-c05e5afd49ad","Type":"ContainerDied","Data":"47628a279e73af6fbc1a00ac015bc98ecb567db83ead186f749f276f23ffeb98"}
Jul 19 15:09:08.059968 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.059810    2585 scope.go:117] "RemoveContainer" containerID="ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"
Jul 19 15:09:08.069707 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.069690    2585 scope.go:117] "RemoveContainer" containerID="001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"
Jul 19 15:09:08.078962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.078940    2585 scope.go:117] "RemoveContainer" containerID="ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"
Jul 19 15:09:08.079260 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:09:08.079228    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc\": container with ID starting with ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc not found: ID does not exist" containerID="ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"
Jul 19 15:09:08.079345 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.079272    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc"} err="failed to get container status \"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc\": rpc error: code = NotFound desc = could not find container \"ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc\": container with ID starting with ee7557b986587675cdb4d380caf9e786d0b3f75e04fa7e30d3f187dfe18effbc not found: ID does not exist"
Jul 19 15:09:08.079345 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.079298    2585 scope.go:117] "RemoveContainer" containerID="001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"
Jul 19 15:09:08.079579 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:09:08.079559    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d\": container with ID starting with 001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d not found: ID does not exist" containerID="001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"
Jul 19 15:09:08.079628 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.079585    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d"} err="failed to get container status \"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d\": rpc error: code = NotFound desc = could not find container \"001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d\": container with ID starting with 001625ba179de0c20035899679432a1640e736ff0f054b5b9659c008a910652d not found: ID does not exist"
Jul 19 15:09:08.080927 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.080897    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:09:08.085030 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.085008    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-79a97677-openshift-default-678594fc4f-n4ln8"]
Jul 19 15:09:08.422857 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:08.422750    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" path="/var/lib/kubelet/pods/b25d773c-a786-4fea-bd4a-c05e5afd49ad/volumes"
Jul 19 15:09:30.181659 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.181551    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"]
Jul 19 15:09:30.182287 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182261    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.182287 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182285    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.182423 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182305    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.182423 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182313    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.182520 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182432    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.182520 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.182449    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25d773c-a786-4fea-bd4a-c05e5afd49ad" containerName="istio-proxy"
Jul 19 15:09:30.185856 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.185833    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:30.188667 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.188625    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:09:30.191457 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.191425    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"]
Jul 19 15:09:30.330245 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.330198    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56\" (UID: \"033f3e2f-2d36-4667-8c5e-0ffd1661071d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:30.431037 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.431003    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56\" (UID: \"033f3e2f-2d36-4667-8c5e-0ffd1661071d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:30.431654 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.431633    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56\" (UID: \"033f3e2f-2d36-4667-8c5e-0ffd1661071d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:30.498846 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.498815    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:30.642879 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:30.642853    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"]
Jul 19 15:09:30.645860 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:09:30.645829    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033f3e2f_2d36_4667_8c5e_0ffd1661071d.slice/crio-b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35 WatchSource:0}: Error finding container b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35: Status 404 returned error can't find the container with id b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35
Jul 19 15:09:31.178852 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:31.178749    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56" event={"ID":"033f3e2f-2d36-4667-8c5e-0ffd1661071d","Type":"ContainerStarted","Data":"fc6e9fbf18ef9449e5aa3f7344a274306e37e9df91b4b4ebcfc1a822e61eec8b"}
Jul 19 15:09:31.178852 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:31.178837    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56" event={"ID":"033f3e2f-2d36-4667-8c5e-0ffd1661071d","Type":"ContainerStarted","Data":"b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35"}
Jul 19 15:09:31.194684 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:31.194626    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56" podStartSLOduration=1.194607829 podStartE2EDuration="1.194607829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:09:31.192937127 +0000 UTC m=+1973.619414140" watchObservedRunningTime="2026-07-19 15:09:31.194607829 +0000 UTC m=+1973.621084842"
Jul 19 15:09:32.185916 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:32.185881    2585 generic.go:356] "Generic (PLEG): container finished" podID="033f3e2f-2d36-4667-8c5e-0ffd1661071d" containerID="fc6e9fbf18ef9449e5aa3f7344a274306e37e9df91b4b4ebcfc1a822e61eec8b" exitCode=0
Jul 19 15:09:32.186125 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:32.185971    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56" event={"ID":"033f3e2f-2d36-4667-8c5e-0ffd1661071d","Type":"ContainerDied","Data":"fc6e9fbf18ef9449e5aa3f7344a274306e37e9df91b4b4ebcfc1a822e61eec8b"}
Jul 19 15:09:33.337538 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:33.337513    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:33.465515 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:33.465402    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca\") pod \"033f3e2f-2d36-4667-8c5e-0ffd1661071d\" (UID: \"033f3e2f-2d36-4667-8c5e-0ffd1661071d\") "
Jul 19 15:09:33.465757 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:33.465733    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "033f3e2f-2d36-4667-8c5e-0ffd1661071d" (UID: "033f3e2f-2d36-4667-8c5e-0ffd1661071d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:09:33.566996 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:33.566961    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/033f3e2f-2d36-4667-8c5e-0ffd1661071d-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:09:34.196758 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:34.196727    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"
Jul 19 15:09:34.196962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:34.196765    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56" event={"ID":"033f3e2f-2d36-4667-8c5e-0ffd1661071d","Type":"ContainerDied","Data":"b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35"}
Jul 19 15:09:34.196962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:34.196831    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b31a90b08a84635740049a24e3c5b3707e6d574a63a62e07f78a74c0d1ab8a35"
Jul 19 15:09:38.001003 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:38.000960    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:38.102911 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:38.102876    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:40.826551 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:40.826516    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:40.927750 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:40.927716    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:41.068684 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:41.068651    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:45.256995 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:45.256961    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"]
Jul 19 15:09:45.261020 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:45.260987    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8b5a03b-25j56"]
Jul 19 15:09:45.263320 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:45.263294    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:45.412413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:45.412379    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:45.662763 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:45.662673    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:46.152193 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:46.152153    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:46.284876 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:46.284843    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:46.396587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:46.396555    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:46.423135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:46.423051    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="033f3e2f-2d36-4667-8c5e-0ffd1661071d" path="/var/lib/kubelet/pods/033f3e2f-2d36-4667-8c5e-0ffd1661071d/volumes"
Jul 19 15:09:47.798037 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:47.798000    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:53.547258 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:53.547225    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:53.697205 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:53.697098    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:53.780742 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:53.780675    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:53.882696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:53.882611    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:09:54.236691 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:09:54.236660    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:18.992520 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:18.992486    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:20.218741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:20.218710    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:26.586553 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:26.586521    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:26.684913 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:26.684873    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:26.798181 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:26.798142    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:42.704305 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:42.704265    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:48.561099 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:48.561065    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:48.630176 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:48.630134    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:48.706308 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:48.706278    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:10:59.278682 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:10:59.278643    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:00.088067 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:00.087984    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:05.872118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:05.872082    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:05.984702 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:05.984657    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:06.076642 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:06.076606    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:06.171986 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:06.171903    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:06.992413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:06.992379    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:07.086325 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:07.086295    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:07.185221 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:07.185187    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:08.856875 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:08.856841    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:14.694515 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:14.694480    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:14.796399 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:14.796369    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:15.373481 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:15.373450    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:20.910990 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:20.910952    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:20.973454 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:20.973419    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:21.159123 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.159088    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:21.477851 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.477817    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:11:21.478266 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.478253    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="033f3e2f-2d36-4667-8c5e-0ffd1661071d" containerName="revoke-keys"
Jul 19 15:11:21.478318 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.478269    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="033f3e2f-2d36-4667-8c5e-0ffd1661071d" containerName="revoke-keys"
Jul 19 15:11:21.478353 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.478338    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="033f3e2f-2d36-4667-8c5e-0ffd1661071d" containerName="revoke-keys"
Jul 19 15:11:21.481522 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.481503    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.483887 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.483866    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4281d3b3-openshift-default-dockercfg-hpjxm\""
Jul 19 15:11:21.499466 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.499335    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:21.501312 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.501289    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:11:21.611144 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611336 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611336 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611216    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf8xc\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611336 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611276    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611336 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611319    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611478 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611345    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611478 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611360    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611478 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.611568 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.611504    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.673612 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.673573    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:21.712700 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.712662    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.712700 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.712699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.712949 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.712736    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.712949 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.712812    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.712949 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.712878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713090 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713034    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713090 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713078    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf8xc\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713166 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713114    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713166 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713144    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713260 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713161    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713525 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713472    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713666 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713646    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713810 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713716    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.713871 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.713811    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.715230 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.715212    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.715359 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.715344    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.720800 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.720768    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf8xc\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.720903 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.720766    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token\") pod \"e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") " pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.792574 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.792468    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:21.940319 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.940277    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:11:21.941989 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:11:21.941961    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f80112_7bb6_4a2c_878c_fccbd3350c5e.slice/crio-ac1890cf051e23382249a4464a22bbe56e60b540a4139809855c94fe0ed3e265 WatchSource:0}: Error finding container ac1890cf051e23382249a4464a22bbe56e60b540a4139809855c94fe0ed3e265: Status 404 returned error can't find the container with id ac1890cf051e23382249a4464a22bbe56e60b540a4139809855c94fe0ed3e265
Jul 19 15:11:21.944157 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.944121    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:21.944238 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.944213    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:21.944277 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:21.944240    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:22.765218 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:22.765178    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerStarted","Data":"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"}
Jul 19 15:11:22.765443 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:22.765240    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerStarted","Data":"ac1890cf051e23382249a4464a22bbe56e60b540a4139809855c94fe0ed3e265"}
Jul 19 15:11:22.786081 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:22.786025    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" podStartSLOduration=1.786008987 podStartE2EDuration="1.786008987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:11:22.784965802 +0000 UTC m=+2085.211442816" watchObservedRunningTime="2026-07-19 15:11:22.786008987 +0000 UTC m=+2085.212486000"
Jul 19 15:11:22.792984 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:22.792949    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:23.798441 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:23.798410    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:24.728030 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:24.727991    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:24.775892 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:24.775858    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:24.776817 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:24.776774    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:24.952142 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:24.952099    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:25.069022 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:25.068894    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:25.213194 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:25.213159    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:25.510926 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:25.510891    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:25.608702 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:25.608672    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:29.220734 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:29.220686    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:29.885823 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:29.885767    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:29.977272 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:29.977240    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:30.333429 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:30.333380    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:34.833886 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.833850    2585 generic.go:356] "Generic (PLEG): container finished" podID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerID="f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c" exitCode=0
Jul 19 15:11:34.834314 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.833923    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerDied","Data":"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"}
Jul 19 15:11:34.834404 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.834345    2585 scope.go:117] "RemoveContainer" containerID="f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"
Jul 19 15:11:34.835325 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.835298    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:34.835380 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.835365    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:34.835416 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:34.835399    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:11:35.843301 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:35.843252    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerStarted","Data":"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"}
Jul 19 15:11:35.892187 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:35.892151    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:35.952418 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:35.952369    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:36.793754 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:36.793713    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:36.794769 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:36.794737    2585 patch_prober.go:28] interesting pod/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 19 15:11:36.795125 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:36.794825    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 19 15:11:37.793766 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:37.793727    2585 patch_prober.go:28] interesting pod/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 19 15:11:37.794199 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:37.793812    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 19 15:11:38.281661 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:38.281628    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:11:38.286914 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:38.286889    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:11:38.793051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:38.793013    2585 patch_prober.go:28] interesting pod/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 19 15:11:38.793247 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:38.793069    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 19 15:11:39.776103 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:39.776065    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:39.797442 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:39.797411    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:39.875092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:39.875061    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:11:46.651563 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:46.651511    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:51.337817 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:51.337762    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:51.721512 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:51.721399    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:57.415650 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.415596    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:57.516363 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.516322    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:57.669761 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.669672    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:11:57.669989 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.669954    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy" containerID="cri-o://edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503" gracePeriod=30
Jul 19 15:11:57.690022 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.689992    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:57.930428 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:57.930346    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:58.049052 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:58.049019    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:58.151719 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:58.151678    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:11:58.274425 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:11:58.274391    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:01.870768 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:01.870731    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:02.931846 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.931816    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:12:02.994895 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.994858    2585 generic.go:356] "Generic (PLEG): container finished" podID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerID="edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503" exitCode=0
Jul 19 15:12:02.995074 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.994901    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"
Jul 19 15:12:02.995074 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.994911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerDied","Data":"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"}
Jul 19 15:12:02.995074 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.994966    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx" event={"ID":"79f80112-7bb6-4a2c-878c-fccbd3350c5e","Type":"ContainerDied","Data":"ac1890cf051e23382249a4464a22bbe56e60b540a4139809855c94fe0ed3e265"}
Jul 19 15:12:02.995074 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:02.994991    2585 scope.go:117] "RemoveContainer" containerID="edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"
Jul 19 15:12:03.004087 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.004072    2585 scope.go:117] "RemoveContainer" containerID="f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"
Jul 19 15:12:03.012098 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.012077    2585 scope.go:117] "RemoveContainer" containerID="edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"
Jul 19 15:12:03.012378 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:12:03.012360    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503\": container with ID starting with edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503 not found: ID does not exist" containerID="edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"
Jul 19 15:12:03.012429 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.012388    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503"} err="failed to get container status \"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503\": rpc error: code = NotFound desc = could not find container \"edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503\": container with ID starting with edab05b30ff42a9177da89e768817adc067d70a7b3e007c2c38fb15a16d64503 not found: ID does not exist"
Jul 19 15:12:03.012429 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.012405    2585 scope.go:117] "RemoveContainer" containerID="f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"
Jul 19 15:12:03.012607 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:12:03.012593    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c\": container with ID starting with f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c not found: ID does not exist" containerID="f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"
Jul 19 15:12:03.012657 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.012612    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c"} err="failed to get container status \"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c\": rpc error: code = NotFound desc = could not find container \"f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c\": container with ID starting with f59ae841e72dd7ccd1c80aa30121b1a92b3bccda164e744113a58c767ec7be1c not found: ID does not exist"
Jul 19 15:12:03.029099 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029076    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029113    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029158    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029175    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029214    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029270    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029321    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf8xc\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029521 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029477    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:12:03.029640 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029521    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029640 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029568    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket\") pod \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\" (UID: \"79f80112-7bb6-4a2c-878c-fccbd3350c5e\") "
Jul 19 15:12:03.029640 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029572    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:12:03.029853 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029803    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:12:03.029928 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029907    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:12:03.030000 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.029982    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.030055 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.030007    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.030055 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.030021    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.030153 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.030088    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:12:03.031498 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.031476    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc" (OuterVolumeSpecName: "kube-api-access-hf8xc") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "kube-api-access-hf8xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:12:03.031587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.031567    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:12:03.031726 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.031712    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:12:03.031828 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.031810    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79f80112-7bb6-4a2c-878c-fccbd3350c5e" (UID: "79f80112-7bb6-4a2c-878c-fccbd3350c5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:12:03.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.130943    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.130990    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf8xc\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-kube-api-access-hf8xc\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.131002    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.131012    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.130997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.131021    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.131321 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.131029    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f80112-7bb6-4a2c-878c-fccbd3350c5e-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:12:03.318042 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.318011    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:12:03.325600 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:03.325571    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4281d3b3-openshift-default-5b9898b6bc-sgbcx"]
Jul 19 15:12:04.053771 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:04.053739    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:04.159624 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:04.159570    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:04.254338 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:04.254300    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:04.419744 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:04.419655    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:04.424056 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:04.424028    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" path="/var/lib/kubelet/pods/79f80112-7bb6-4a2c-878c-fccbd3350c5e/volumes"
Jul 19 15:12:10.155630 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:10.155591    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:10.228015 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:10.227973    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:16.994890 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:16.994853    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:17.287950 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.287737    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:12:17.288812 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.288757    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:12:17.288812 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.288803    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:12:17.288972 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.288958    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:12:17.292573 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.292556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.295428 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.295409    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c515b9b9-openshift-default-dockercfg-d8jxp\""
Jul 19 15:12:17.304668 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.304646    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:12:17.316329 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.316301    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:17.352876 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.352843    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.352876 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.352879    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.352897    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.352918    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.352983    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.353005    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.353039    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.353057    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.353118 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.353082    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlwk\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.444971 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.444935    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:17.454187 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454156    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454197    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454219    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454244    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454274    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454298    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454332    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454658 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454356    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454658 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454401    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlwk\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.454931 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454907    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.455044 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.454959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.455044 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.455006    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.455147 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.455119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.455192 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.455180    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.456857 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.456837    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.456962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.456934    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.461931 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.461903    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.462294 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.462271    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlwk\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk\") pod \"e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") " pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.604236 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.604145    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:17.755013 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.754980    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:12:17.756175 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:12:17.756139    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef23fd3e_7111_44e0_8ab7_fdcc513605e6.slice/crio-1d0b4bcf5706fd470b6ce717d4b3bd89f9b42e61e7c4270813284acf1625b405 WatchSource:0}: Error finding container 1d0b4bcf5706fd470b6ce717d4b3bd89f9b42e61e7c4270813284acf1625b405: Status 404 returned error can't find the container with id 1d0b4bcf5706fd470b6ce717d4b3bd89f9b42e61e7c4270813284acf1625b405
Jul 19 15:12:17.758661 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.758617    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:17.758738 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.758718    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:17.758798 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:17.758765    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:18.079858 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:18.079824    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerStarted","Data":"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"}
Jul 19 15:12:18.080497 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:18.079890    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerStarted","Data":"1d0b4bcf5706fd470b6ce717d4b3bd89f9b42e61e7c4270813284acf1625b405"}
Jul 19 15:12:18.099724 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:18.099671    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podStartSLOduration=1.099654212 podStartE2EDuration="1.099654212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:12:18.097762222 +0000 UTC m=+2140.524239225" watchObservedRunningTime="2026-07-19 15:12:18.099654212 +0000 UTC m=+2140.526131225"
Jul 19 15:12:18.604848 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:18.604804    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:18.609889 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:18.609865    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:19.085248 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:19.085215    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:19.086100 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:19.086074    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:19.995646 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:19.995617    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:20.089668 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:20.089627    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:20.236305 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:20.236273    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:20.431523 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:20.431439    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:20.790806 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:20.790754    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:20.892998 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:20.892968    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:24.503049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:24.503013    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:33.789013 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:33.788972    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:12:34.914126 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:34.912895    2585 patch_prober.go:28] interesting pod/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:40706->10.134.0.56:15021: read: connection reset by peer" start-of-body=
Jul 19 15:12:34.914126 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:34.912974    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:40706->10.134.0.56:15021: read: connection reset by peer"
Jul 19 15:12:35.189343 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.189241    2585 generic.go:356] "Generic (PLEG): container finished" podID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerID="233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3" exitCode=0
Jul 19 15:12:35.189343 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.189285    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerDied","Data":"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"}
Jul 19 15:12:35.189656 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.189642    2585 scope.go:117] "RemoveContainer" containerID="233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"
Jul 19 15:12:35.190718 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.190690    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:35.190771 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.190763    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:35.190846 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:35.190820    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:12:36.195992 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:36.195961    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerStarted","Data":"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"}
Jul 19 15:12:36.604769 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:36.604733    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:36.604994 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:36.604939    2585 patch_prober.go:28] interesting pod/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 19 15:12:36.605075 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:36.605011    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 19 15:12:37.605445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:37.605401    2585 patch_prober.go:28] interesting pod/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 19 15:12:37.605965 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:37.605464    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 19 15:12:38.604668 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:38.604630    2585 patch_prober.go:28] interesting pod/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 19 15:12:38.604867 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:38.604708    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 19 15:12:39.604743 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:39.604705    2585 patch_prober.go:28] interesting pod/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 19 15:12:39.605173 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:39.604766    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 19 15:12:40.608410 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:40.608378    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:40.608856 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:40.608730    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:40.609482 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:40.609466    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:12:59.560999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:12:59.560963    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:08.265709 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:08.265675    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:12.844840 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:12.844753    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:15.118280 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:15.118249    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:20.149151 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:20.149118    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:26.003933 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:26.003897    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:46.279885 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:46.279849    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:56.244499 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:56.244468    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:13:57.011356 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:13:57.011318    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:02.446032 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.445955    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:02.546201 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.546166    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:02.749233 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.749201    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:14:02.749425 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.749388    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy" containerID="cri-o://7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976" gracePeriod=30
Jul 19 15:14:02.788667 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.788631    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:02.978353 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:02.978311    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:04.168459 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:04.168428    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:04.268016 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:04.267983    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:04.368054 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:04.368024    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:05.123716 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:05.123673    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:05.221631 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:05.221595    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:05.322135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:05.322100    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:06.715689 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:06.715654    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:08.015348 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.015283    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:14:08.128493 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128447    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128524    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128554    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nlwk\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128581    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128602    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128642    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.128714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128683    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128744    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128812    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data\") pod \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\" (UID: \"ef23fd3e-7111-44e0-8ab7-fdcc513605e6\") "
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128951    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.128960    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129058    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:14:08.129092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129084    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:14:08.129407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129225    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.129407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129250    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.129407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129263    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.129407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129276    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.129693 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.129673    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:14:08.130968 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.130926    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:14:08.131101 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.130996    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:14:08.131246 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.131230    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:14:08.131291 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.131271    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk" (OuterVolumeSpecName: "kube-api-access-2nlwk") pod "ef23fd3e-7111-44e0-8ab7-fdcc513605e6" (UID: "ef23fd3e-7111-44e0-8ab7-fdcc513605e6"). InnerVolumeSpecName "kube-api-access-2nlwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:14:08.230483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.230448    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.230483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.230479    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.230483 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.230488    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.230722 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.230497    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.230722 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.230505    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nlwk\" (UniqueName: \"kubernetes.io/projected/ef23fd3e-7111-44e0-8ab7-fdcc513605e6-kube-api-access-2nlwk\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:14:08.690032 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.689936    2585 generic.go:356] "Generic (PLEG): container finished" podID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerID="7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976" exitCode=0
Jul 19 15:14:08.690032 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.689989    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"
Jul 19 15:14:08.690032 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.690015    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerDied","Data":"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"}
Jul 19 15:14:08.690306 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.690069    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9" event={"ID":"ef23fd3e-7111-44e0-8ab7-fdcc513605e6","Type":"ContainerDied","Data":"1d0b4bcf5706fd470b6ce717d4b3bd89f9b42e61e7c4270813284acf1625b405"}
Jul 19 15:14:08.690306 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.690098    2585 scope.go:117] "RemoveContainer" containerID="7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"
Jul 19 15:14:08.699257 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.699237    2585 scope.go:117] "RemoveContainer" containerID="233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"
Jul 19 15:14:08.707658 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.707628    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:14:08.707869 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.707831    2585 scope.go:117] "RemoveContainer" containerID="7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"
Jul 19 15:14:08.708081 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:14:08.708064    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976\": container with ID starting with 7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976 not found: ID does not exist" containerID="7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"
Jul 19 15:14:08.708137 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.708089    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976"} err="failed to get container status \"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976\": rpc error: code = NotFound desc = could not find container \"7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976\": container with ID starting with 7141ff53a87066545c2105146562a6fd68c34871be0103e83158d21090dcd976 not found: ID does not exist"
Jul 19 15:14:08.708137 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.708104    2585 scope.go:117] "RemoveContainer" containerID="233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"
Jul 19 15:14:08.708460 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:14:08.708426    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3\": container with ID starting with 233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3 not found: ID does not exist" containerID="233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"
Jul 19 15:14:08.708585 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.708460    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3"} err="failed to get container status \"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3\": rpc error: code = NotFound desc = could not find container \"233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3\": container with ID starting with 233d8dcfd8b3a7ec558b75912e6d8626929f070974b72adf19a5eb00815f5ae3 not found: ID does not exist"
Jul 19 15:14:08.710877 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:08.710854    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c515b9b9-openshift-default-54d8dd9b47-knwl9"]
Jul 19 15:14:10.422936 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:10.422895    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" path="/var/lib/kubelet/pods/ef23fd3e-7111-44e0-8ab7-fdcc513605e6/volumes"
Jul 19 15:14:12.610109 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:12.610060    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:12.710331 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:12.710276    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:13.290270 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:13.290238    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:19.629600 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:19.629567    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:19.693921 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:19.693879    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:19.779297 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:19.779261    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:19.869360 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:19.869324    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:30.336976 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:30.336941    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:30.689459 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:30.689373    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:38.392395 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:38.392362    2585 scope.go:117] "RemoveContainer" containerID="bda44bac40065c4e0bca624d997d31372253455750ad851169d2dea879841c31"
Jul 19 15:14:38.711428 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:38.711309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:38.812456 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:38.812413    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:51.679426 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:51.679379    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:57.476699 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:57.476662    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:57.610334 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:57.610297    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:57.841281 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:57.841155    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:58.141756 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.141662    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:14:58.142241 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142220    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:14:58.142241 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142242    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:14:58.142420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142273    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:14:58.142420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142283    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:14:58.142420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142408    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:14:58.142639 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142425    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f80112-7bb6-4a2c-878c-fccbd3350c5e" containerName="istio-proxy"
Jul 19 15:14:58.142639 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.142435    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:14:58.145804 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.145769    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.149047 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.149029    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f72db2-openshift-default-dockercfg-bvc42\""
Jul 19 15:14:58.158877 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.158854    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:14:58.172764 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.172734    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:58.303824 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303762    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.303824 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303829    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303861    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303883    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303907    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.303941    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7zjh\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.304008    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304073 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.304029    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.304345 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.304146    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.348592 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.348558    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:14:58.405166 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405166 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405231    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405283    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405313    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405351    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405579 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405402    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405579 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405534    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405581    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405821 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405797    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405913 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405837    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7zjh\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.405974 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.405914    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.406154 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.406130    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.406244 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.406226    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.407764 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.407723    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.408014 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.407994    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.413156 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.413132    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.413734 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.413717    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7zjh\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh\") pod \"e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") " pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.457963 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.457928    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:58.608561 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.608530    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:14:58.610755 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:14:58.610720    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc49d6d_5e40_4034_846b_308de0265ff4.slice/crio-6d0768b52b1148cc27ea00f93472b8e9b9e7e9b8877d4227603a4a37ae09ea30 WatchSource:0}: Error finding container 6d0768b52b1148cc27ea00f93472b8e9b9e7e9b8877d4227603a4a37ae09ea30: Status 404 returned error can't find the container with id 6d0768b52b1148cc27ea00f93472b8e9b9e7e9b8877d4227603a4a37ae09ea30
Jul 19 15:14:58.613052 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.613017    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:14:58.613145 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.613084    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:14:58.613145 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.613115    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:14:58.963085 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.962997    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerStarted","Data":"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"}
Jul 19 15:14:58.963085 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.963047    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerStarted","Data":"6d0768b52b1148cc27ea00f93472b8e9b9e7e9b8877d4227603a4a37ae09ea30"}
Jul 19 15:14:58.982274 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:58.982226    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podStartSLOduration=0.98221103 podStartE2EDuration="982.21103ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:14:58.97997462 +0000 UTC m=+2301.406451647" watchObservedRunningTime="2026-07-19 15:14:58.98221103 +0000 UTC m=+2301.408688042"
Jul 19 15:14:59.458419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:59.458379    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:59.464065 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:59.464040    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:59.967936 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:59.967910    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:14:59.968916 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:14:59.968895    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:15:00.930985 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:00.930947    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:01.030342 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:01.030309    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:01.151429 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:01.151396    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:01.424152 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:01.424104    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:01.590840 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:01.590776    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:01.691981 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:01.691898    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:05.442236 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:05.442204    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:05.776147 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:05.776109    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:06.085261 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:06.085172    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:06.581166 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:06.581125    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:08.526541 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:08.526506    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:08.612931 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:08.612891    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:08.744369 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:08.744336    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:09.069326 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:09.069294    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:09.180169 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:09.180135    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:09.276323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:09.276294    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:13.028745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:13.028711    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:16.009005 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:16.008953    2585 patch_prober.go:28] interesting pod/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 19 15:15:16.009488 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:16.009062    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 19 15:15:17.062419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.062385    2585 generic.go:356] "Generic (PLEG): container finished" podID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerID="f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84" exitCode=0
Jul 19 15:15:17.062926 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.062445    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerDied","Data":"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"}
Jul 19 15:15:17.062984 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.062934    2585 scope.go:117] "RemoveContainer" containerID="f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"
Jul 19 15:15:17.064016 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.063983    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:15:17.064069 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.064054    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:15:17.064101 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:17.064086    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:15:18.072565 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:18.072521    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerStarted","Data":"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"}
Jul 19 15:15:18.459133 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:18.459035    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:15:18.459473 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:18.459449    2585 patch_prober.go:28] interesting pod/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:15:18.459638 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:18.459612    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:15:19.458568 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:19.458526    2585 patch_prober.go:28] interesting pod/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:15:19.459226 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:19.459190    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:15:20.459522 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:20.459477    2585 patch_prober.go:28] interesting pod/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:15:20.460175 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:20.460137    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:15:21.458901 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:21.458853    2585 patch_prober.go:28] interesting pod/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 19 15:15:21.459128 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:21.458929    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 19 15:15:22.432766 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:22.432727    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:22.462372 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:22.462341    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:15:22.462839 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:22.462815    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:15:22.463521 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:22.463503    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:15:22.504056 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:22.504015    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:29.937058 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:29.936951    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:30.039421 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:30.039379    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:32.885661 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:32.885603    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:15:38.416152 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:38.416112    2585 scope.go:117] "RemoveContainer" containerID="fc6e9fbf18ef9449e5aa3f7344a274306e37e9df91b4b4ebcfc1a822e61eec8b"
Jul 19 15:15:55.018752 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:15:55.018710    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:32.803564 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:32.803529    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:34.023612 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.023578    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:34.133427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.133390    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"]
Jul 19 15:16:34.133898 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.133882    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:16:34.133965 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.133901    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef23fd3e-7111-44e0-8ab7-fdcc513605e6" containerName="istio-proxy"
Jul 19 15:16:34.137407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.137390    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:34.139978 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.139738    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:16:34.141956 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.141929    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"]
Jul 19 15:16:34.239253 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.239213    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f\" (UID: \"f753427c-4753-4acd-b64d-ef107c370cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:34.340493 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.340381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f\" (UID: \"f753427c-4753-4acd-b64d-ef107c370cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:34.341314 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.341275    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f\" (UID: \"f753427c-4753-4acd-b64d-ef107c370cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:34.452886 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.452853    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:34.590445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.590387    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"]
Jul 19 15:16:34.593340 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:16:34.593249    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf753427c_4753_4acd_b64d_ef107c370cd9.slice/crio-bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c WatchSource:0}: Error finding container bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c: Status 404 returned error can't find the container with id bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c
Jul 19 15:16:34.695190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.695155    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:34.796337 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:34.796301    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:35.499191 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:35.499157    2585 generic.go:356] "Generic (PLEG): container finished" podID="f753427c-4753-4acd-b64d-ef107c370cd9" containerID="e13c4df4686bafef96952d2f3f1df52878be64f4a3f3a4d88ba258d63bd6fefd" exitCode=0
Jul 19 15:16:35.499583 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:35.499243    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f" event={"ID":"f753427c-4753-4acd-b64d-ef107c370cd9","Type":"ContainerDied","Data":"e13c4df4686bafef96952d2f3f1df52878be64f4a3f3a4d88ba258d63bd6fefd"}
Jul 19 15:16:35.499583 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:35.499294    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f" event={"ID":"f753427c-4753-4acd-b64d-ef107c370cd9","Type":"ContainerStarted","Data":"bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c"}
Jul 19 15:16:36.649914 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:36.649888    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:36.772450 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:36.772348    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca\") pod \"f753427c-4753-4acd-b64d-ef107c370cd9\" (UID: \"f753427c-4753-4acd-b64d-ef107c370cd9\") "
Jul 19 15:16:36.772720 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:36.772694    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f753427c-4753-4acd-b64d-ef107c370cd9" (UID: "f753427c-4753-4acd-b64d-ef107c370cd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:16:36.873320 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:36.873281    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f753427c-4753-4acd-b64d-ef107c370cd9-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:16:37.510208 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:37.510181    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"
Jul 19 15:16:37.510387 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:37.510225    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f" event={"ID":"f753427c-4753-4acd-b64d-ef107c370cd9","Type":"ContainerDied","Data":"bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c"}
Jul 19 15:16:37.510387 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:37.510275    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcf2d76778bd4218bef81a241770dc4c05099825a7711e810f2a89a06667a74c"
Jul 19 15:16:38.313479 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:38.313449    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:16:38.320921 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:38.320897    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:16:44.521476 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:44.521442    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:49.210044 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:49.209988    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:49.228842 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:49.228815    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"]
Jul 19 15:16:49.230663 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:49.230642    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f72db2-4dq9f"]
Jul 19 15:16:49.608973 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:49.608942    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:50.422964 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:50.422926    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f753427c-4753-4acd-b64d-ef107c370cd9" path="/var/lib/kubelet/pods/f753427c-4753-4acd-b64d-ef107c370cd9/volumes"
Jul 19 15:16:55.424027 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.423989    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:55.524935 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.524879    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:55.685702 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.685609    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:16:55.686199 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.686163    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy" containerID="cri-o://e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5" gracePeriod=30
Jul 19 15:16:55.705732 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.705683    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:55.939985 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:55.939872    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:56.057310 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.057276    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"]
Jul 19 15:16:56.057732 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.057719    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f753427c-4753-4acd-b64d-ef107c370cd9" containerName="revoke-keys"
Jul 19 15:16:56.057806 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.057735    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f753427c-4753-4acd-b64d-ef107c370cd9" containerName="revoke-keys"
Jul 19 15:16:56.057840 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.057817    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f753427c-4753-4acd-b64d-ef107c370cd9" containerName="revoke-keys"
Jul 19 15:16:56.061256 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.061240    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:56.063487 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.063459    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:16:56.067438 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.067410    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k\" (UID: \"d965c754-b773-4d03-a992-8a8269dc08de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:56.067707 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.067681    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"]
Jul 19 15:16:56.168875 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.168842    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k\" (UID: \"d965c754-b773-4d03-a992-8a8269dc08de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:56.169568 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.169545    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k\" (UID: \"d965c754-b773-4d03-a992-8a8269dc08de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:56.375003 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.374971    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:56.518460 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.518365    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"]
Jul 19 15:16:56.522035 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:16:56.522000    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd965c754_b773_4d03_a992_8a8269dc08de.slice/crio-f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385 WatchSource:0}: Error finding container f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385: Status 404 returned error can't find the container with id f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385
Jul 19 15:16:56.556737 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.556706    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:56.611714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.611678    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k" event={"ID":"d965c754-b773-4d03-a992-8a8269dc08de","Type":"ContainerStarted","Data":"f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385"}
Jul 19 15:16:56.655537 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:56.655446    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:16:57.618413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:57.618319    2585 generic.go:356] "Generic (PLEG): container finished" podID="d965c754-b773-4d03-a992-8a8269dc08de" containerID="ca04a3f5b22bd339273b21f7fb4de613fc52d7d11368e0493fc2fcfeb0d481ce" exitCode=0
Jul 19 15:16:57.618413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:57.618387    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k" event={"ID":"d965c754-b773-4d03-a992-8a8269dc08de","Type":"ContainerDied","Data":"ca04a3f5b22bd339273b21f7fb4de613fc52d7d11368e0493fc2fcfeb0d481ce"}
Jul 19 15:16:58.766339 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:58.766313    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:58.794997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:58.794968    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca\") pod \"d965c754-b773-4d03-a992-8a8269dc08de\" (UID: \"d965c754-b773-4d03-a992-8a8269dc08de\") "
Jul 19 15:16:58.795313 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:58.795288    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d965c754-b773-4d03-a992-8a8269dc08de" (UID: "d965c754-b773-4d03-a992-8a8269dc08de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:16:58.896461 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:58.896365    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d965c754-b773-4d03-a992-8a8269dc08de-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:16:59.629196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:59.629166    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"
Jul 19 15:16:59.629374 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:59.629166    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k" event={"ID":"d965c754-b773-4d03-a992-8a8269dc08de","Type":"ContainerDied","Data":"f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385"}
Jul 19 15:16:59.629374 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:16:59.629291    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f889ffba03284f6d425c8da0da8b2bb5cb403b367ac8d44bcb20baaec3e44385"
Jul 19 15:17:00.965854 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:00.965826    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:17:01.017915 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.017879    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018132 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.017985    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018132 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018063    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018132 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018110    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018319 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018151    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018319 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018214    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7zjh\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018319 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018280    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018482 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018331    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018482 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018375    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert\") pod \"0bc49d6d-5e40-4034-846b-308de0265ff4\" (UID: \"0bc49d6d-5e40-4034-846b-308de0265ff4\") "
Jul 19 15:17:01.018482 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018389    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:17:01.019115 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018711    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.019115 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.019065    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:17:01.019435 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.018231    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:17:01.019657 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.019559    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data" (OuterVolumeSpecName: "istio-data") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:17:01.019657 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.019631    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:17:01.020977 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.020940    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:17:01.021078 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.021020    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token" (OuterVolumeSpecName: "istio-token") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:17:01.021228 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.021205    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:17:01.021807 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.021748    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh" (OuterVolumeSpecName: "kube-api-access-q7zjh") pod "0bc49d6d-5e40-4034-846b-308de0265ff4" (UID: "0bc49d6d-5e40-4034-846b-308de0265ff4"). InnerVolumeSpecName "kube-api-access-q7zjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:17:01.119401 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119362    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119401 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119395    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bc49d6d-5e40-4034-846b-308de0265ff4-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119401 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119403    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119401 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119411    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119673 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119421    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119673 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119429    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119673 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119438    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7zjh\" (UniqueName: \"kubernetes.io/projected/0bc49d6d-5e40-4034-846b-308de0265ff4-kube-api-access-q7zjh\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.119673 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.119446    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bc49d6d-5e40-4034-846b-308de0265ff4-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:01.640831 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.640777    2585 generic.go:356] "Generic (PLEG): container finished" podID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerID="e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5" exitCode=0
Jul 19 15:17:01.641051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.640841    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"
Jul 19 15:17:01.641051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.640856    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerDied","Data":"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"}
Jul 19 15:17:01.641051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.640914    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk" event={"ID":"0bc49d6d-5e40-4034-846b-308de0265ff4","Type":"ContainerDied","Data":"6d0768b52b1148cc27ea00f93472b8e9b9e7e9b8877d4227603a4a37ae09ea30"}
Jul 19 15:17:01.641051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.640943    2585 scope.go:117] "RemoveContainer" containerID="e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"
Jul 19 15:17:01.651031 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.651011    2585 scope.go:117] "RemoveContainer" containerID="f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"
Jul 19 15:17:01.659337 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.659301    2585 scope.go:117] "RemoveContainer" containerID="e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"
Jul 19 15:17:01.659846 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:17:01.659817    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5\": container with ID starting with e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5 not found: ID does not exist" containerID="e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"
Jul 19 15:17:01.660001 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.659856    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5"} err="failed to get container status \"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5\": rpc error: code = NotFound desc = could not find container \"e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5\": container with ID starting with e9837bdeb6663f34b1e004372a36026d0d8bb5d986374a782c05fd8017ec62b5 not found: ID does not exist"
Jul 19 15:17:01.660001 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.659883    2585 scope.go:117] "RemoveContainer" containerID="f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"
Jul 19 15:17:01.660227 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:17:01.660195    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84\": container with ID starting with f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84 not found: ID does not exist" containerID="f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"
Jul 19 15:17:01.660390 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.660362    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84"} err="failed to get container status \"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84\": rpc error: code = NotFound desc = could not find container \"f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84\": container with ID starting with f26667ed3a87fa977811f3796f260b31a93789307df953046a22c6392a339f84 not found: ID does not exist"
Jul 19 15:17:01.662361 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.662340    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:17:01.665918 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:01.665896    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f72db2-openshift-default-6dfc6df8c5-sjnqk"]
Jul 19 15:17:02.422848 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:02.422814    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" path="/var/lib/kubelet/pods/0bc49d6d-5e40-4034-846b-308de0265ff4/volumes"
Jul 19 15:17:06.613026 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:06.612990    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:11.131082 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:11.131044    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"]
Jul 19 15:17:11.133797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:11.133757    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:11.133925 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:11.133906    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-36dc1d-pwh8k"]
Jul 19 15:17:11.551589 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:11.551555    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:12.422419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:12.422384    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d965c754-b773-4d03-a992-8a8269dc08de" path="/var/lib/kubelet/pods/d965c754-b773-4d03-a992-8a8269dc08de/volumes"
Jul 19 15:17:17.295602 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:17.295569    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:17.396089 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:17.396056    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:17.577247 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:17.577172    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:17.815402 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:17.815343    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:18.163796 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.163747    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:18.446854 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.446735    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:17:18.448015 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.447989    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d965c754-b773-4d03-a992-8a8269dc08de" containerName="revoke-keys"
Jul 19 15:17:18.448177 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448163    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d965c754-b773-4d03-a992-8a8269dc08de" containerName="revoke-keys"
Jul 19 15:17:18.448290 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448277    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:18.448386 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448368    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:18.448386 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448390    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:18.448570 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448399    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:18.448570 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448526    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:18.448570 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.448544    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d965c754-b773-4d03-a992-8a8269dc08de" containerName="revoke-keys"
Jul 19 15:17:18.451709 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.451686    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.453959 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.453934    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b6c0bd-openshift-default-dockercfg-pt2bn\""
Jul 19 15:17:18.461179 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.461156    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:17:18.474922 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.474897    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:18.591258 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591215    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591258 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591255    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591276    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591375    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591426    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591458    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591625 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591625 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591542    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7jrf\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.591625 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.591600    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.692681 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692636    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.692681 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692679    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.692977 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692699    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.692977 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692724    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.692977 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693144 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.692999    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693144 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693068    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693144 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693099    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7jrf\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693299 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693154    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693299 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693174    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693299 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693192    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693454 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693312    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693454 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693379    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.693612 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.693585    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.695440 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.695411    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.695649 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.695630    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.703760 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.703703    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.708514 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.708490    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7jrf\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf\") pod \"e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") " pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.763053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.763013    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:18.789385 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.789357    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:18.908204 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.908179    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:17:18.909814 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:17:18.909771    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92b9022_4ef7_4952_b65d_cb635d5fcaef.slice/crio-7da24e774e410a0263a769ae7175919e7f37511988b815199186d6e80273e790 WatchSource:0}: Error finding container 7da24e774e410a0263a769ae7175919e7f37511988b815199186d6e80273e790: Status 404 returned error can't find the container with id 7da24e774e410a0263a769ae7175919e7f37511988b815199186d6e80273e790
Jul 19 15:17:18.911920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.911888    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:18.911991 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.911951    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:18.911991 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:18.911979    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:19.739175 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:19.739140    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerStarted","Data":"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"}
Jul 19 15:17:19.739643 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:19.739189    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerStarted","Data":"7da24e774e410a0263a769ae7175919e7f37511988b815199186d6e80273e790"}
Jul 19 15:17:19.762600 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:19.762551    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podStartSLOduration=1.762533732 podStartE2EDuration="1.762533732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:17:19.757685224 +0000 UTC m=+2442.184162237" watchObservedRunningTime="2026-07-19 15:17:19.762533732 +0000 UTC m=+2442.189010744"
Jul 19 15:17:19.763257 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:19.763230    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:19.768351 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:19.768323    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:20.744619 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:20.744584    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:20.745885 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:20.745864    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:21.670980 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:21.670943    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:21.738981 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:21.738945    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:21.810959 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:21.810920    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:22.041820 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:22.041755    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:22.229126 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:22.229083    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:22.329437 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:22.329344    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:26.189462 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:26.189431    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:26.680270 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:26.680232    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:26.997319 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:26.997283    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:27.353839 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:27.353724    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:30.282821 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:30.282769    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:30.384917 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:30.384875    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:30.638751 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:30.638674    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:31.004128 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:31.004088    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:31.196057 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:31.196003    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:31.293006 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:31.292919    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:34.706506 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:34.706474    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:36.787032 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:36.786979    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 19 15:17:36.787464 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:36.787052    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 19 15:17:37.836150 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.836115    2585 generic.go:356] "Generic (PLEG): container finished" podID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerID="e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30" exitCode=0
Jul 19 15:17:37.836698 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.836165    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerDied","Data":"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"}
Jul 19 15:17:37.836698 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.836662    2585 scope.go:117] "RemoveContainer" containerID="e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"
Jul 19 15:17:37.837889 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.837860    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:37.837975 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.837930    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:37.837975 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:37.837965    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:17:38.847310 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:38.847274    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerStarted","Data":"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"}
Jul 19 15:17:39.764275 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:39.764226    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:39.764502 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:39.764400    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:17:39.764502 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:39.764444    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:17:40.764035 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:40.763985    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:17:40.764657 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:40.764062    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:17:41.764891 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:41.764389    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:17:41.764891 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:41.764460    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:17:42.763697 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.763652    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 19 15:17:42.763945 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.763721    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 19 15:17:42.816340 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.816306    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"]
Jul 19 15:17:42.816834 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.816818    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc49d6d-5e40-4034-846b-308de0265ff4" containerName="istio-proxy"
Jul 19 15:17:42.822108 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.822080    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:42.824754 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.824725    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:17:42.827131 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.827103    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"]
Jul 19 15:17:42.928136 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:42.928098    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd\" (UID: \"5775c682-9ed2-4815-9602-59422794b580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:43.029874 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.029739    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd\" (UID: \"5775c682-9ed2-4815-9602-59422794b580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:43.030397 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.030373    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd\" (UID: \"5775c682-9ed2-4815-9602-59422794b580\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:43.134754 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.134717    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:43.321063 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.321039    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"]
Jul 19 15:17:43.323618 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:17:43.323588    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5775c682_9ed2_4815_9602_59422794b580.slice/crio-9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b WatchSource:0}: Error finding container 9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b: Status 404 returned error can't find the container with id 9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b
Jul 19 15:17:43.337832 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.337803    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:43.439297 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.439263    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:43.767265 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.767232    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:43.767518 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.767498    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:43.768165 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.768149    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:17:43.883614 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.883574    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd" event={"ID":"5775c682-9ed2-4815-9602-59422794b580","Type":"ContainerStarted","Data":"985217a783f1dc6c5d35ed7bc04729a340c789d6427f8eb8e44e642325a78c7a"}
Jul 19 15:17:43.884069 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.883636    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd" event={"ID":"5775c682-9ed2-4815-9602-59422794b580","Type":"ContainerStarted","Data":"9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b"}
Jul 19 15:17:43.901270 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:43.901170    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd" podStartSLOduration=1.9011547960000001 podStartE2EDuration="1.901154796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:17:43.899041841 +0000 UTC m=+2466.325518854" watchObservedRunningTime="2026-07-19 15:17:43.901154796 +0000 UTC m=+2466.327631808"
Jul 19 15:17:44.896467 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:44.896429    2585 generic.go:356] "Generic (PLEG): container finished" podID="5775c682-9ed2-4815-9602-59422794b580" containerID="985217a783f1dc6c5d35ed7bc04729a340c789d6427f8eb8e44e642325a78c7a" exitCode=0
Jul 19 15:17:44.896893 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:44.896516    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd" event={"ID":"5775c682-9ed2-4815-9602-59422794b580","Type":"ContainerDied","Data":"985217a783f1dc6c5d35ed7bc04729a340c789d6427f8eb8e44e642325a78c7a"}
Jul 19 15:17:46.043419 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.043389    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:46.160722 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.160641    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca\") pod \"5775c682-9ed2-4815-9602-59422794b580\" (UID: \"5775c682-9ed2-4815-9602-59422794b580\") "
Jul 19 15:17:46.161006 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.160983    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5775c682-9ed2-4815-9602-59422794b580" (UID: "5775c682-9ed2-4815-9602-59422794b580"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:17:46.261858 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.261825    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5775c682-9ed2-4815-9602-59422794b580-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:17:46.907849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.907817    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd" event={"ID":"5775c682-9ed2-4815-9602-59422794b580","Type":"ContainerDied","Data":"9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b"}
Jul 19 15:17:46.907849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.907845    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"
Jul 19 15:17:46.908064 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:46.907873    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9abd2233a38a4270e146a8311e62a26348ecad6493240f3bd86a194d43c5240b"
Jul 19 15:17:51.935075 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:51.935026    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:52.034505 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:52.034468    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:53.432489 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:53.432450    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:53.532706 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:53.532654    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:17:58.662370 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:17:58.662336    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:02.891634 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:02.891597    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"]
Jul 19 15:18:02.895367 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:02.895341    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6c0bd-c4ghd"]
Jul 19 15:18:02.897760 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:02.897737    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:03.276251 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:03.276219    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:04.272457 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:04.272422    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:04.422769 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:04.422736    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5775c682-9ed2-4815-9602-59422794b580" path="/var/lib/kubelet/pods/5775c682-9ed2-4815-9602-59422794b580/volumes"
Jul 19 15:18:08.876662 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:08.876611    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:08.936594 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:08.936562    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:09.114103 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.114064    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:18:09.114331 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.114297    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" containerID="cri-o://9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9" gracePeriod=30
Jul 19 15:18:09.132179 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.132094    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:09.366269 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.366231    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:09.478196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.478164    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"]
Jul 19 15:18:09.478873 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.478856    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5775c682-9ed2-4815-9602-59422794b580" containerName="revoke-keys"
Jul 19 15:18:09.478922 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.478875    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5775c682-9ed2-4815-9602-59422794b580" containerName="revoke-keys"
Jul 19 15:18:09.478957 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.478951    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5775c682-9ed2-4815-9602-59422794b580" containerName="revoke-keys"
Jul 19 15:18:09.482406 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.482390    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:09.484821 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.484798    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:18:09.487982 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.487956    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"]
Jul 19 15:18:09.591038 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.591000    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5a578a-2w79j\" (UID: \"09224b9a-722b-4b18-8223-53571b961fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:09.692006 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.691959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5a578a-2w79j\" (UID: \"09224b9a-722b-4b18-8223-53571b961fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:09.692608 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.692588    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5a578a-2w79j\" (UID: \"09224b9a-722b-4b18-8223-53571b961fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:09.795063 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.794975    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:09.997966 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:09.997929    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:10.099254 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:10.099171    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:10.142293 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:10.142262    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"]
Jul 19 15:18:10.145450 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:18:10.145380    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09224b9a_722b_4b18_8223_53571b961fe2.slice/crio-c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854 WatchSource:0}: Error finding container c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854: Status 404 returned error can't find the container with id c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854
Jul 19 15:18:11.041129 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:11.041096    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j" event={"ID":"09224b9a-722b-4b18-8223-53571b961fe2","Type":"ContainerStarted","Data":"d3af3aaa6aca464106cf3f69c9679556da0e6cfff0d6d17d20567645cef98423"}
Jul 19 15:18:11.041608 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:11.041155    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j" event={"ID":"09224b9a-722b-4b18-8223-53571b961fe2","Type":"ContainerStarted","Data":"c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854"}
Jul 19 15:18:11.058110 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:11.058044    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j" podStartSLOduration=2.058026435 podStartE2EDuration="2.058026435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:18:11.055896245 +0000 UTC m=+2493.482373261" watchObservedRunningTime="2026-07-19 15:18:11.058026435 +0000 UTC m=+2493.484503447"
Jul 19 15:18:12.048239 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:12.048195    2585 generic.go:356] "Generic (PLEG): container finished" podID="09224b9a-722b-4b18-8223-53571b961fe2" containerID="d3af3aaa6aca464106cf3f69c9679556da0e6cfff0d6d17d20567645cef98423" exitCode=0
Jul 19 15:18:12.048627 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:12.048279    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j" event={"ID":"09224b9a-722b-4b18-8223-53571b961fe2","Type":"ContainerDied","Data":"d3af3aaa6aca464106cf3f69c9679556da0e6cfff0d6d17d20567645cef98423"}
Jul 19 15:18:13.196708 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.196679    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:13.220648 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.220612    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca\") pod \"09224b9a-722b-4b18-8223-53571b961fe2\" (UID: \"09224b9a-722b-4b18-8223-53571b961fe2\") "
Jul 19 15:18:13.221046 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.221020    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09224b9a-722b-4b18-8223-53571b961fe2" (UID: "09224b9a-722b-4b18-8223-53571b961fe2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:18:13.322254 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.322168    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09224b9a-722b-4b18-8223-53571b961fe2-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:13.770615 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.770578    2585 patch_prober.go:28] interesting pod/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:18:13.770768 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:13.770644    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:18:14.060531 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.060439    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"
Jul 19 15:18:14.060531 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.060457    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j" event={"ID":"09224b9a-722b-4b18-8223-53571b961fe2","Type":"ContainerDied","Data":"c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854"}
Jul 19 15:18:14.060531 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.060508    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c339da59b8234c498cfcb9a592fa86d56f7cba459a4538b029cb73a25acb5854"
Jul 19 15:18:14.376893 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.376869    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:18:14.429895 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.429865    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.429914    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.429940    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430014    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7jrf\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430046    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430132    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430132    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430159    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430209    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430240    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket\") pod \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\" (UID: \"e92b9022-4ef7-4952-b65d-cb635d5fcaef\") "
Jul 19 15:18:14.430323 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.430252    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:18:14.431203 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431047    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:18:14.431203 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431166    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431367    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data" (OuterVolumeSpecName: "istio-data") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431669    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431693    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431707    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431720    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.431755 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.431734    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.432606 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.432569    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf" (OuterVolumeSpecName: "kube-api-access-q7jrf") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "kube-api-access-q7jrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:18:14.432714 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.432646    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:18:14.432775 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.432740    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token" (OuterVolumeSpecName: "istio-token") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:18:14.433044 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.433024    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e92b9022-4ef7-4952-b65d-cb635d5fcaef" (UID: "e92b9022-4ef7-4952-b65d-cb635d5fcaef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:18:14.532273 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.532237    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7jrf\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-kube-api-access-q7jrf\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.532273 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.532269    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.532273 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.532279    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:14.532506 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:14.532288    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e92b9022-4ef7-4952-b65d-cb635d5fcaef-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:18:15.067972 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.067939    2585 generic.go:356] "Generic (PLEG): container finished" podID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerID="9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9" exitCode=0
Jul 19 15:18:15.068135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.067984    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"
Jul 19 15:18:15.068135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.067987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerDied","Data":"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"}
Jul 19 15:18:15.068135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.068071    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474" event={"ID":"e92b9022-4ef7-4952-b65d-cb635d5fcaef","Type":"ContainerDied","Data":"7da24e774e410a0263a769ae7175919e7f37511988b815199186d6e80273e790"}
Jul 19 15:18:15.068135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.068103    2585 scope.go:117] "RemoveContainer" containerID="9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"
Jul 19 15:18:15.077920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.077904    2585 scope.go:117] "RemoveContainer" containerID="e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"
Jul 19 15:18:15.085951 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.085931    2585 scope.go:117] "RemoveContainer" containerID="9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"
Jul 19 15:18:15.086198 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:18:15.086176    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9\": container with ID starting with 9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9 not found: ID does not exist" containerID="9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"
Jul 19 15:18:15.086251 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.086207    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9"} err="failed to get container status \"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9\": rpc error: code = NotFound desc = could not find container \"9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9\": container with ID starting with 9cb1ae0bc376cb0d200351b7a7116520a3efed045c9c43da55ce139aec2b67f9 not found: ID does not exist"
Jul 19 15:18:15.086251 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.086224    2585 scope.go:117] "RemoveContainer" containerID="e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"
Jul 19 15:18:15.086492 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:18:15.086460    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30\": container with ID starting with e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30 not found: ID does not exist" containerID="e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"
Jul 19 15:18:15.086531 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.086498    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30"} err="failed to get container status \"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30\": rpc error: code = NotFound desc = could not find container \"e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30\": container with ID starting with e9346d7b3801dd78c29bda03f8bec0c1e4ec7fad482b60db4d722c8ab8a23c30 not found: ID does not exist"
Jul 19 15:18:15.090035 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.090012    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:18:15.093929 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:15.093907    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b6c0bd-openshift-default-77cd5c8dd9-5j474"]
Jul 19 15:18:16.423117 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:16.423069    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" path="/var/lib/kubelet/pods/e92b9022-4ef7-4952-b65d-cb635d5fcaef/volumes"
Jul 19 15:18:24.890988 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:24.890948    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:29.553826 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:29.553790    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"]
Jul 19 15:18:29.557644 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:29.557610    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5a578a-2w79j"]
Jul 19 15:18:29.558053 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:29.558026    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:29.940078 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:29.939960    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:30.423536 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:30.423498    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09224b9a-722b-4b18-8223-53571b961fe2" path="/var/lib/kubelet/pods/09224b9a-722b-4b18-8223-53571b961fe2/volumes"
Jul 19 15:18:35.693132 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:35.693090    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:35.793834 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:35.793797    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:35.976289 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:35.976256    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:36.201901 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.201870    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:36.564146 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.564110    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:36.854756 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.854672    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:18:36.855350 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855329    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09224b9a-722b-4b18-8223-53571b961fe2" containerName="revoke-keys"
Jul 19 15:18:36.855445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855354    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09224b9a-722b-4b18-8223-53571b961fe2" containerName="revoke-keys"
Jul 19 15:18:36.855445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855390    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:18:36.855445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855401    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:18:36.855445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855415    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:18:36.855445 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855424    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:18:36.855688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855538    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="09224b9a-722b-4b18-8223-53571b961fe2" containerName="revoke-keys"
Jul 19 15:18:36.855688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.855556    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:18:36.860407 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.860382    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.862918 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.862896    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-7679e0-openshift-default-dockercfg-dwg99\""
Jul 19 15:18:36.871405 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.871372    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:18:36.883599 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.883562    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:36.948803 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948739    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.948999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948847    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.948999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948875    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.948999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948929    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.948999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948956    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.948999 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.948997    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv8tz\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.949260 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.949047    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.949260 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.949070    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:36.949260 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:36.949141    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.026727 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.026690    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:37.050154 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050116    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050154 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050153    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050186    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050244    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050284    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050309    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050360    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050427 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050388    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050429    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv8tz\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050624    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050741 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050702    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.050937 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.050850    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.051051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.051027    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.051211 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.051191    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.052808 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.052770    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.053055 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.053037    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.059405 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.059382    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.059503 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.059404    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv8tz\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz\") pod \"e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") " pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.173639 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.173534    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:37.343529 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.343494    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:18:37.345903 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:18:37.345865    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59da23b_6f10_4eb5_a632_0e5c358feca5.slice/crio-e68dab85cc6fab21bc60876cb4655589e0950461ce0d6025be56ac38e132eae9 WatchSource:0}: Error finding container e68dab85cc6fab21bc60876cb4655589e0950461ce0d6025be56ac38e132eae9: Status 404 returned error can't find the container with id e68dab85cc6fab21bc60876cb4655589e0950461ce0d6025be56ac38e132eae9
Jul 19 15:18:37.348357 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.348319    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:37.348429 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.348412    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:37.348466 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:37.348455    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:38.199972 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:38.199933    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerStarted","Data":"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"}
Jul 19 15:18:38.200382 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:38.199989    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerStarted","Data":"e68dab85cc6fab21bc60876cb4655589e0950461ce0d6025be56ac38e132eae9"}
Jul 19 15:18:38.223140 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:38.223084    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podStartSLOduration=2.223068272 podStartE2EDuration="2.223068272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:18:38.220070881 +0000 UTC m=+2520.646547897" watchObservedRunningTime="2026-07-19 15:18:38.223068272 +0000 UTC m=+2520.649545340"
Jul 19 15:18:39.174240 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:39.174201    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:39.179281 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:39.179252    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:39.204886 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:39.204850    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:39.205831 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:39.205810    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:40.154836 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:40.154802    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.304962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:40.304926    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.495607 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:40.495577    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.594846 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:40.594814    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:40.956100 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:40.956010    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:41.059738 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:41.059705    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:44.677582 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:44.677548    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:45.010369 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:45.010339    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:45.465563 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:45.465475    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:45.906255 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:45.906214    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:48.605553 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:48.605524    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:48.680861 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:48.680827    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:48.784792 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:48.784744    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:49.371884 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:49.371818    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:49.469465 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:49.469428    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:53.113730 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:53.113690    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:18:54.206668 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.206633    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:18:54.207131 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.206703    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:18:54.301315 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.301279    2585 generic.go:356] "Generic (PLEG): container finished" podID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerID="b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4" exitCode=0
Jul 19 15:18:54.301503 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.301325    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerDied","Data":"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"}
Jul 19 15:18:54.301841 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.301818    2585 scope.go:117] "RemoveContainer" containerID="b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"
Jul 19 15:18:54.302847 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.302820    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:54.302922 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.302891    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:54.302964 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:54.302924    2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 19 15:18:55.307876 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:55.307839    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerStarted","Data":"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"}
Jul 19 15:18:56.173932 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:56.173895    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:18:56.174203 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:56.174071    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:18:56.174203 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:56.174146    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:18:57.174848 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:57.174815    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:18:57.175272 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:57.174879    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:18:58.174906 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:58.174869    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:18:58.175307 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:58.174928    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:18:59.174963 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:59.174925    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 19 15:18:59.175372 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:18:59.174984    2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 19 15:19:00.177938 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:00.177906    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:19:00.178346 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:00.178250    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:19:00.179064 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:00.179046    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:19:01.853552 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:01.853518    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:01.953587 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:01.953554    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:10.183969 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:10.183933    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:10.286059 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:10.286030    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:10.931378 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:10.931344    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:11.068827 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:11.068764    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:39.428653 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:39.428609    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:39.525729 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:39.525699    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:46.787489 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:46.787443    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:46.838625 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:46.838591    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:55.302843 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:55.302810    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:19:59.604289 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:19:59.604247    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:00.406280 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:20:00.406247    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:20:44.298345 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:20:44.298311    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:21:38.349632 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:21:38.349547    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:21:38.357613 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:21:38.357589    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:22:18.543726 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:18.543676    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:19.611989 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:19.611954    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:32.892201 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.892165    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:22:32.892825 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.892807    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92b9022-4ef7-4952-b65d-cb635d5fcaef" containerName="istio-proxy"
Jul 19 15:22:32.896006 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.895989    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.899255 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.899218    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7679e0\"/\"sub-test-model-7679e0-kserve-self-signed-certs\""
Jul 19 15:22:32.899398 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.899300    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7679e0\"/\"kube-root-ca.crt\""
Jul 19 15:22:32.899398 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.899313    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7679e0\"/\"openshift-service-ca.crt\""
Jul 19 15:22:32.899398 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.899393    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7679e0\"/\"default-dockercfg-dcp4j\""
Jul 19 15:22:32.905190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.905164    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:22:32.916248 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916219    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.916418 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916257    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.916418 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.916418 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.916542 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916458    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c75w\" (UniqueName: \"kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.916542 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.916504    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:32.996220 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:32.996184    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:22:33.016978 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.016945    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c75w\" (UniqueName: \"kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017141 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.016997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017141 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017022    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017141 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017044    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017141 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017068    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017349 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017303    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017503 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017475    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017595 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017578    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.017635 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.017596    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.019506 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.019480    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.019671 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.019654    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.024975 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.024949    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c75w\" (UniqueName: \"kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w\") pod \"sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") " pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.206931 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.206838    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:33.348128 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.348103    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:22:33.349983 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:22:33.349958    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod164e3539_2a25_420e_bcf5_ad0ff9bff12a.slice/crio-aa7277b1838a449b5336106da74f16d70976caefb303ca138b09ac9b4107a2ff WatchSource:0}: Error finding container aa7277b1838a449b5336106da74f16d70976caefb303ca138b09ac9b4107a2ff: Status 404 returned error can't find the container with id aa7277b1838a449b5336106da74f16d70976caefb303ca138b09ac9b4107a2ff
Jul 19 15:22:33.497550 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.497514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" event={"ID":"164e3539-2a25-420e-bcf5-ad0ff9bff12a","Type":"ContainerStarted","Data":"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"}
Jul 19 15:22:33.497550 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.497560    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" event={"ID":"164e3539-2a25-420e-bcf5-ad0ff9bff12a","Type":"ContainerStarted","Data":"aa7277b1838a449b5336106da74f16d70976caefb303ca138b09ac9b4107a2ff"}
Jul 19 15:22:33.513286 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:33.513234    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" podStartSLOduration=1.513217836 podStartE2EDuration="1.513217836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:22:33.511324828 +0000 UTC m=+2755.937801842" watchObservedRunningTime="2026-07-19 15:22:33.513217836 +0000 UTC m=+2755.939694848"
Jul 19 15:22:38.499705 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:38.499676    2585 scope.go:117] "RemoveContainer" containerID="e13c4df4686bafef96952d2f3f1df52878be64f4a3f3a4d88ba258d63bd6fefd"
Jul 19 15:22:43.207386 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:43.207346    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:43.207875 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:43.207406    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:43.220420 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:43.220391    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:22:43.569181 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:22:43.569155    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:23:00.721462 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:23:00.721350    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:38.521048 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:23:38.521013    2585 scope.go:117] "RemoveContainer" containerID="ca04a3f5b22bd339273b21f7fb4de613fc52d7d11368e0493fc2fcfeb0d481ce"
Jul 19 15:23:50.683623 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:23:50.683581    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:23:50.700867 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:23:50.700832    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:23:51.625063 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:23:51.625026    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:01.924658 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:01.924615    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:02.860886 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:02.860845    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:02.961849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:02.961809    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:05.706693 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:05.706653    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" podUID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" containerName="main" containerID="cri-o://59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962" gracePeriod=45
Jul 19 15:24:05.969702 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:05.969643    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:24:06.002569 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.002539    2585 generic.go:356] "Generic (PLEG): container finished" podID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" containerID="59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962" exitCode=0
Jul 19 15:24:06.002745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.002589    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"
Jul 19 15:24:06.002745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.002594    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" event={"ID":"164e3539-2a25-420e-bcf5-ad0ff9bff12a","Type":"ContainerDied","Data":"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"}
Jul 19 15:24:06.002745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.002708    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl" event={"ID":"164e3539-2a25-420e-bcf5-ad0ff9bff12a","Type":"ContainerDied","Data":"aa7277b1838a449b5336106da74f16d70976caefb303ca138b09ac9b4107a2ff"}
Jul 19 15:24:06.002745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.002735    2585 scope.go:117] "RemoveContainer" containerID="59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"
Jul 19 15:24:06.011637 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.011619    2585 scope.go:117] "RemoveContainer" containerID="59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"
Jul 19 15:24:06.011902 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:24:06.011881    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962\": container with ID starting with 59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962 not found: ID does not exist" containerID="59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"
Jul 19 15:24:06.011962 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.011916    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962"} err="failed to get container status \"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962\": rpc error: code = NotFound desc = could not find container \"59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962\": container with ID starting with 59df75d9c98c2e33b3066aa8a0039eb94c480ae3530825d7ff1921b39fb9d962 not found: ID does not exist"
Jul 19 15:24:06.047708 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047676    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.047849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047720    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.047849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047753    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.047849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047778    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c75w\" (UniqueName: \"kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.047849 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047831    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.048071 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.047853    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home\") pod \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\" (UID: \"164e3539-2a25-420e-bcf5-ad0ff9bff12a\") "
Jul 19 15:24:06.048071 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.048018    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache" (OuterVolumeSpecName: "model-cache") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:06.048173 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.048068    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:06.048230 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.048213    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home" (OuterVolumeSpecName: "home") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:06.048296 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.048279    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-model-cache\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.048350 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.048302    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tmp-dir\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.049983 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.049959    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm" (OuterVolumeSpecName: "dshm") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:24:06.050079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.050046    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:24:06.050079 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.050059    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w" (OuterVolumeSpecName: "kube-api-access-4c75w") pod "164e3539-2a25-420e-bcf5-ad0ff9bff12a" (UID: "164e3539-2a25-420e-bcf5-ad0ff9bff12a"). InnerVolumeSpecName "kube-api-access-4c75w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:24:06.149212 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.149181    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-home\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.149212 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.149208    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/164e3539-2a25-420e-bcf5-ad0ff9bff12a-tls-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.149212 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.149220    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c75w\" (UniqueName: \"kubernetes.io/projected/164e3539-2a25-420e-bcf5-ad0ff9bff12a-kube-api-access-4c75w\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.149450 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.149228    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/164e3539-2a25-420e-bcf5-ad0ff9bff12a-dshm\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:24:06.324129 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.324093    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:24:06.327240 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.327213    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7679e0/sub-test-model-7679e0-kserve-6864c9f7f5-mr2gl"]
Jul 19 15:24:06.425172 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:06.425137    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" path="/var/lib/kubelet/pods/164e3539-2a25-420e-bcf5-ad0ff9bff12a/volumes"
Jul 19 15:24:10.885749 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:10.885714    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:10.987390 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:10.987352    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:21.722970 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:21.722930    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:38.543380 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:38.543299    2585 scope.go:117] "RemoveContainer" containerID="d3af3aaa6aca464106cf3f69c9679556da0e6cfff0d6d17d20567645cef98423"
Jul 19 15:24:38.552543 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:38.552523    2585 scope.go:117] "RemoveContainer" containerID="985217a783f1dc6c5d35ed7bc04729a340c789d6427f8eb8e44e642325a78c7a"
Jul 19 15:24:40.417161 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:40.417118    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:24:40.610551 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:24:40.610515    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:02.448305 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:02.448266    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:03.437498 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:03.437462    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:07.653113 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.653077    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140663706091568"]
Jul 19 15:25:07.653527 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.653514    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" containerName="main"
Jul 19 15:25:07.653527 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.653528    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" containerName="main"
Jul 19 15:25:07.653606 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.653602    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="164e3539-2a25-420e-bcf5-ad0ff9bff12a" containerName="main"
Jul 19 15:25:07.657014 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.656996    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:07.660097 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.660073    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9sx9t\""
Jul 19 15:25:07.662072 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.662047    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140663706091568"]
Jul 19 15:25:07.752169 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.752129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjq8n\" (UniqueName: \"kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n\") pod \"test-curl-7292-140663706091568\" (UID: \"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2\") " pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:07.853107 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.853069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjq8n\" (UniqueName: \"kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n\") pod \"test-curl-7292-140663706091568\" (UID: \"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2\") " pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:07.860914 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.860880    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjq8n\" (UniqueName: \"kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n\") pod \"test-curl-7292-140663706091568\" (UID: \"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2\") " pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:07.968648 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:07.968556    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:08.102496 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:08.102463    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140663706091568"]
Jul 19 15:25:08.104686 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:25:08.104657    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd939ce4a_6b9e_47a4_ad70_0d441b8da2c2.slice/crio-bdab9291a2e053db5d082c55aa655b5d3e14eeb5a84e783784622492f17a034d WatchSource:0}: Error finding container bdab9291a2e053db5d082c55aa655b5d3e14eeb5a84e783784622492f17a034d: Status 404 returned error can't find the container with id bdab9291a2e053db5d082c55aa655b5d3e14eeb5a84e783784622492f17a034d
Jul 19 15:25:08.106052 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:08.106032    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:25:08.333048 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:08.333012    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140663706091568" event={"ID":"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2","Type":"ContainerStarted","Data":"bdab9291a2e053db5d082c55aa655b5d3e14eeb5a84e783784622492f17a034d"}
Jul 19 15:25:09.340928 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:09.340893    2585 generic.go:356] "Generic (PLEG): container finished" podID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" containerID="5fb107a3fcfc6e6150c2f72640b986a72d785d998dc7a542de53fdec945772ad" exitCode=0
Jul 19 15:25:09.341352 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:09.340986    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140663706091568" event={"ID":"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2","Type":"ContainerDied","Data":"5fb107a3fcfc6e6150c2f72640b986a72d785d998dc7a542de53fdec945772ad"}
Jul 19 15:25:10.492972 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.492943    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:10.511996 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.511869    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140663706091568_d939ce4a-6b9e-47a4-ad70-0d441b8da2c2/test-curl-7292-140663706091568/0.log"
Jul 19 15:25:10.535462 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.535425    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140663706091568"]
Jul 19 15:25:10.537387 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.537366    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140663706091568"]
Jul 19 15:25:10.681417 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.681331    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjq8n\" (UniqueName: \"kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n\") pod \"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2\" (UID: \"d939ce4a-6b9e-47a4-ad70-0d441b8da2c2\") "
Jul 19 15:25:10.683586 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.683557    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n" (OuterVolumeSpecName: "kube-api-access-jjq8n") pod "d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" (UID: "d939ce4a-6b9e-47a4-ad70-0d441b8da2c2"). InnerVolumeSpecName "kube-api-access-jjq8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:25:10.783036 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:10.782994    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjq8n\" (UniqueName: \"kubernetes.io/projected/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2-kube-api-access-jjq8n\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:11.353067 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:11.353028    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:11.353294 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:11.353028    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdab9291a2e053db5d082c55aa655b5d3e14eeb5a84e783784622492f17a034d"
Jul 19 15:25:11.363737 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:25:11.363702    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140663706091568\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" pod="opendatahub/test-curl-7292-140663706091568"
Jul 19 15:25:12.424321 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:12.424283    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" path="/var/lib/kubelet/pods/d939ce4a-6b9e-47a4-ad70-0d441b8da2c2/volumes"
Jul 19 15:25:16.692974 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.692938    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140663705906320"]
Jul 19 15:25:16.693382 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.693363    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" containerName="test-curl-7292-140663706091568"
Jul 19 15:25:16.693382 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.693382    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" containerName="test-curl-7292-140663706091568"
Jul 19 15:25:16.693462 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.693458    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d939ce4a-6b9e-47a4-ad70-0d441b8da2c2" containerName="test-curl-7292-140663706091568"
Jul 19 15:25:16.696760 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.696743    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:16.699706 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.699679    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9sx9t\""
Jul 19 15:25:16.701912 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.701880    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140663705906320"]
Jul 19 15:25:16.737935 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.737902    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nqkf\" (UniqueName: \"kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf\") pod \"test-curl-7292-140663705906320\" (UID: \"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2\") " pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:16.838731 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.838690    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nqkf\" (UniqueName: \"kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf\") pod \"test-curl-7292-140663705906320\" (UID: \"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2\") " pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:16.847045 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:16.847014    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nqkf\" (UniqueName: \"kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf\") pod \"test-curl-7292-140663705906320\" (UID: \"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2\") " pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:17.007217 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:17.007176    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:17.360158 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:17.360125    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140663705906320"]
Jul 19 15:25:17.362095 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:25:17.362058    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6e58f2c_fdcd_44fa_a48b_c56ef29878d2.slice/crio-cad088d5d51f5becc9f31961b266bc36f0fe9d6c14004b577c853ce6096c873c WatchSource:0}: Error finding container cad088d5d51f5becc9f31961b266bc36f0fe9d6c14004b577c853ce6096c873c: Status 404 returned error can't find the container with id cad088d5d51f5becc9f31961b266bc36f0fe9d6c14004b577c853ce6096c873c
Jul 19 15:25:17.387481 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:17.387449    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140663705906320" event={"ID":"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2","Type":"ContainerStarted","Data":"cad088d5d51f5becc9f31961b266bc36f0fe9d6c14004b577c853ce6096c873c"}
Jul 19 15:25:18.394479 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:18.394443    2585 generic.go:356] "Generic (PLEG): container finished" podID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" containerID="6318ef1edb7b2e8058593207ffef4e2b570b24587e8ae0383a843ca189e2a5e4" exitCode=0
Jul 19 15:25:18.394932 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:18.394491    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140663705906320" event={"ID":"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2","Type":"ContainerDied","Data":"6318ef1edb7b2e8058593207ffef4e2b570b24587e8ae0383a843ca189e2a5e4"}
Jul 19 15:25:19.549099 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.549065    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:19.564520 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.564491    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nqkf\" (UniqueName: \"kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf\") pod \"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2\" (UID: \"e6e58f2c-fdcd-44fa-a48b-c56ef29878d2\") "
Jul 19 15:25:19.566559 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.566524    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf" (OuterVolumeSpecName: "kube-api-access-6nqkf") pod "e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" (UID: "e6e58f2c-fdcd-44fa-a48b-c56ef29878d2"). InnerVolumeSpecName "kube-api-access-6nqkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:25:19.581504 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.581479    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140663705906320_e6e58f2c-fdcd-44fa-a48b-c56ef29878d2/test-curl-7292-140663705906320/0.log"
Jul 19 15:25:19.617074 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.617044    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140663705906320"]
Jul 19 15:25:19.623009 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.622984    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140663705906320"]
Jul 19 15:25:19.665834 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:19.665740    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nqkf\" (UniqueName: \"kubernetes.io/projected/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2-kube-api-access-6nqkf\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:20.405126 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:20.405097    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:20.405126 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:20.405126    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cad088d5d51f5becc9f31961b266bc36f0fe9d6c14004b577c853ce6096c873c"
Jul 19 15:25:20.415203 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:25:20.415174    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140663705906320\" is forbidden: User \"system:node:ip-10-0-141-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-105.ec2.internal' and this object" podUID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" pod="opendatahub/test-curl-7292-140663705906320"
Jul 19 15:25:20.423180 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:20.423150    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" path="/var/lib/kubelet/pods/e6e58f2c-fdcd-44fa-a48b-c56ef29878d2/volumes"
Jul 19 15:25:22.802580 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.802540    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"]
Jul 19 15:25:22.802997 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.802983    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" containerName="test-curl-7292-140663705906320"
Jul 19 15:25:22.803051 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.802998    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" containerName="test-curl-7292-140663705906320"
Jul 19 15:25:22.803102 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.803091    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6e58f2c-fdcd-44fa-a48b-c56ef29878d2" containerName="test-curl-7292-140663705906320"
Jul 19 15:25:22.806554 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.806538    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:22.809619 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.809600    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:25:22.818498 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.818470    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"]
Jul 19 15:25:22.898364 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.898315    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t\" (UID: \"62f21c8a-0b84-4762-977e-1d01af420ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:22.999055 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.999013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t\" (UID: \"62f21c8a-0b84-4762-977e-1d01af420ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:22.999670 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:22.999652    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t\" (UID: \"62f21c8a-0b84-4762-977e-1d01af420ed8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:23.116926 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:23.116843    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:23.253206 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:23.253174    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"]
Jul 19 15:25:23.254239 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:25:23.254215    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f21c8a_0b84_4762_977e_1d01af420ed8.slice/crio-3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc WatchSource:0}: Error finding container 3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc: Status 404 returned error can't find the container with id 3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc
Jul 19 15:25:23.385684 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:23.385586    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:23.422711 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:23.422679    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t" event={"ID":"62f21c8a-0b84-4762-977e-1d01af420ed8","Type":"ContainerStarted","Data":"3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc"}
Jul 19 15:25:23.487057 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:23.487022    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:24.429064 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:24.429026    2585 generic.go:356] "Generic (PLEG): container finished" podID="62f21c8a-0b84-4762-977e-1d01af420ed8" containerID="406701959e74693c8fd9b5cb5b14b8ef593dac1726c6f98098d448a400627be2" exitCode=0
Jul 19 15:25:24.429467 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:24.429110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t" event={"ID":"62f21c8a-0b84-4762-977e-1d01af420ed8","Type":"ContainerDied","Data":"406701959e74693c8fd9b5cb5b14b8ef593dac1726c6f98098d448a400627be2"}
Jul 19 15:25:25.577696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:25.577668    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:25.623647 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:25.623613    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca\") pod \"62f21c8a-0b84-4762-977e-1d01af420ed8\" (UID: \"62f21c8a-0b84-4762-977e-1d01af420ed8\") "
Jul 19 15:25:25.624034 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:25.624010    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62f21c8a-0b84-4762-977e-1d01af420ed8" (UID: "62f21c8a-0b84-4762-977e-1d01af420ed8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:25:25.725109 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:25.725017    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f21c8a-0b84-4762-977e-1d01af420ed8-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:26.441127 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:26.441095    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t" event={"ID":"62f21c8a-0b84-4762-977e-1d01af420ed8","Type":"ContainerDied","Data":"3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc"}
Jul 19 15:25:26.441127 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:26.441126    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"
Jul 19 15:25:26.441333 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:26.441152    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f880126e988d7ab4dc19b22f5d128f0fecd7724bf7f3bc384e8baeacc9871cc"
Jul 19 15:25:33.197024 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:33.196981    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:37.885001 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:37.884966    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:37.908187 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:37.908152    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"]
Jul 19 15:25:37.912447 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:37.912423    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7679e0-kzs8t"]
Jul 19 15:25:38.305207 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:38.305168    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:38.423554 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:38.423524    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f21c8a-0b84-4762-977e-1d01af420ed8" path="/var/lib/kubelet/pods/62f21c8a-0b84-4762-977e-1d01af420ed8/volumes"
Jul 19 15:25:44.236122 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.236080    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:44.339344 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.339305    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:44.486347 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.486254    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:25:44.486554 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.486529    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" containerID="cri-o://045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03" gracePeriod=30
Jul 19 15:25:44.504220 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.504188    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:44.802266 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.802175    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:44.881299 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.881266    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"]
Jul 19 15:25:44.881719 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.881706    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f21c8a-0b84-4762-977e-1d01af420ed8" containerName="revoke-keys"
Jul 19 15:25:44.881765 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.881720    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f21c8a-0b84-4762-977e-1d01af420ed8" containerName="revoke-keys"
Jul 19 15:25:44.881825 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.881802    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f21c8a-0b84-4762-977e-1d01af420ed8" containerName="revoke-keys"
Jul 19 15:25:44.887107 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.887085    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:44.889426 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.889394    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bq7rt\""
Jul 19 15:25:44.891920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:44.891895    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"]
Jul 19 15:25:45.016749 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.016716    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz\" (UID: \"bfc96968-844d-418c-9821-79f8d2f9c0fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:45.118092 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.117990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz\" (UID: \"bfc96968-844d-418c-9821-79f8d2f9c0fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:45.118681 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.118660    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz\" (UID: \"bfc96968-844d-418c-9821-79f8d2f9c0fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:45.180637 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.180600    2585 patch_prober.go:28] interesting pod/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 19 15:25:45.180829 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.180674    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 19 15:25:45.200347 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.200323    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:45.340372 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.340130    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"]
Jul 19 15:25:45.342723 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:25:45.342691    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc96968_844d_418c_9821_79f8d2f9c0fa.slice/crio-3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296 WatchSource:0}: Error finding container 3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296: Status 404 returned error can't find the container with id 3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296
Jul 19 15:25:45.546318 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.546285    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:45.548476 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.548445    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz" event={"ID":"bfc96968-844d-418c-9821-79f8d2f9c0fa","Type":"ContainerStarted","Data":"3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296"}
Jul 19 15:25:45.652105 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:45.652011    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:46.555140 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:46.555106    2585 generic.go:356] "Generic (PLEG): container finished" podID="bfc96968-844d-418c-9821-79f8d2f9c0fa" containerID="fc51d6718d5bba70060a29bcb43081ef3ea63a5bb9aa36fd047f8e03a0f83aa7" exitCode=0
Jul 19 15:25:46.555563 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:46.555202    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz" event={"ID":"bfc96968-844d-418c-9821-79f8d2f9c0fa","Type":"ContainerDied","Data":"fc51d6718d5bba70060a29bcb43081ef3ea63a5bb9aa36fd047f8e03a0f83aa7"}
Jul 19 15:25:47.702049 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:47.702023    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:47.842287 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:47.842198    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca\") pod \"bfc96968-844d-418c-9821-79f8d2f9c0fa\" (UID: \"bfc96968-844d-418c-9821-79f8d2f9c0fa\") "
Jul 19 15:25:47.842560 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:47.842539    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bfc96968-844d-418c-9821-79f8d2f9c0fa" (UID: "bfc96968-844d-418c-9821-79f8d2f9c0fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:25:47.943298 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:47.943259    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfc96968-844d-418c-9821-79f8d2f9c0fa-maas-api-service-ca\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:48.567117 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:48.567090    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"
Jul 19 15:25:48.567295 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:48.567081    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz" event={"ID":"bfc96968-844d-418c-9821-79f8d2f9c0fa","Type":"ContainerDied","Data":"3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296"}
Jul 19 15:25:48.567295 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:48.567213    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a2f6eeb6d31f25f1dbc76a681ecf1481efab9d7523b58a60f5d0e1008c4f296"
Jul 19 15:25:49.746691 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.746667    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:25:49.864413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864333    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864367    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864386    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864427    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv8tz\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864484    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864602    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:49.864696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864621    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864696 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864669    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864988 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864732    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864988 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864757    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token\") pod \"e59da23b-6f10-4eb5-a632-0e5c358feca5\" (UID: \"e59da23b-6f10-4eb5-a632-0e5c358feca5\") "
Jul 19 15:25:49.864988 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.864919    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:25:49.865139 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865050    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:49.865139 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865118    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:49.865241 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865202    2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-certs\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.865241 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865225    2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-workload-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.865241 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865238    2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-credential-socket\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.865354 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865252    2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59da23b-6f10-4eb5-a632-0e5c358feca5-istiod-ca-cert\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.865354 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.865246    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data" (OuterVolumeSpecName: "istio-data") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:49.866828 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.866802    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:49.866936 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.866828    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token" (OuterVolumeSpecName: "istio-token") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:25:49.867108 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.867084    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 19 15:25:49.867169 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.867093    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz" (OuterVolumeSpecName: "kube-api-access-zv8tz") pod "e59da23b-6f10-4eb5-a632-0e5c358feca5" (UID: "e59da23b-6f10-4eb5-a632-0e5c358feca5"). InnerVolumeSpecName "kube-api-access-zv8tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:25:49.966084 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.966046    2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-podinfo\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.966084 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.966070    2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-envoy\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.966084 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.966091    2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-token\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.966290 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.966100    2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59da23b-6f10-4eb5-a632-0e5c358feca5-istio-data\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:49.966290 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:49.966109    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv8tz\" (UniqueName: \"kubernetes.io/projected/e59da23b-6f10-4eb5-a632-0e5c358feca5-kube-api-access-zv8tz\") on node \"ip-10-0-141-105.ec2.internal\" DevicePath \"\""
Jul 19 15:25:50.582015 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.581980    2585 generic.go:356] "Generic (PLEG): container finished" podID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerID="045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03" exitCode=0
Jul 19 15:25:50.582190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.582028    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"
Jul 19 15:25:50.582190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.582068    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerDied","Data":"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"}
Jul 19 15:25:50.582190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.582128    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv" event={"ID":"e59da23b-6f10-4eb5-a632-0e5c358feca5","Type":"ContainerDied","Data":"e68dab85cc6fab21bc60876cb4655589e0950461ce0d6025be56ac38e132eae9"}
Jul 19 15:25:50.582190 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.582155    2585 scope.go:117] "RemoveContainer" containerID="045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"
Jul 19 15:25:50.591223 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.591201    2585 scope.go:117] "RemoveContainer" containerID="b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"
Jul 19 15:25:50.599152 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.599130    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:25:50.599357 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.599346    2585 scope.go:117] "RemoveContainer" containerID="045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"
Jul 19 15:25:50.599605 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:25:50.599589    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03\": container with ID starting with 045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03 not found: ID does not exist" containerID="045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"
Jul 19 15:25:50.599646 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.599611    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03"} err="failed to get container status \"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03\": rpc error: code = NotFound desc = could not find container \"045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03\": container with ID starting with 045e2992af175a72d4e085c7bb974d5c532d3f46d2af8e3167992d946d14cf03 not found: ID does not exist"
Jul 19 15:25:50.599646 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.599626    2585 scope.go:117] "RemoveContainer" containerID="b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"
Jul 19 15:25:50.599898 ip-10-0-141-105 kubenswrapper[2585]: E0719 15:25:50.599850    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4\": container with ID starting with b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4 not found: ID does not exist" containerID="b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"
Jul 19 15:25:50.600034 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.599905    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4"} err="failed to get container status \"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4\": rpc error: code = NotFound desc = could not find container \"b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4\": container with ID starting with b71651c19b6324491e0fb80d68f56512a37e2d881705fca07ba232e363d23ca4 not found: ID does not exist"
Jul 19 15:25:50.602825 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:50.602805    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-7679e0-openshift-default-7d79b94d79-jk6xv"]
Jul 19 15:25:52.423119 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:52.423085    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" path="/var/lib/kubelet/pods/e59da23b-6f10-4eb5-a632-0e5c358feca5/volumes"
Jul 19 15:25:55.403018 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:55.402979    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:59.949135 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:59.949036    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:25:59.949862 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:59.949841    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"]
Jul 19 15:25:59.952957 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:25:59.952934    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3c4e71-9c4kz"]
Jul 19 15:26:00.349332 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:00.349281    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:00.422339 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:00.422308    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc96968-844d-418c-9821-79f8d2f9c0fa" path="/var/lib/kubelet/pods/bfc96968-844d-418c-9821-79f8d2f9c0fa/volumes"
Jul 19 15:26:06.142216 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:06.142184    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:06.242870 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:06.242833    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:06.414318 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:06.414220    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:06.664745 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:06.664651    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:07.059397 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:07.059361    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:11.575754 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:11.575718    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:12.485083 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:12.485046    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gmvds_d8de0306-00ee-4873-81ed-4306db0c034a/manager/0.log"
Jul 19 15:26:12.613819 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:12.613776    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5khfl_f813f876-72df-41ae-b2cf-a3ad0f598b7b/manager/0.log"
Jul 19 15:26:12.991831 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:12.991802    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qk9cl_c00bb303-832b-484b-87f5-1ffe043c81d6/manager/0.log"
Jul 19 15:26:13.337327 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:13.337242    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6_104c28db-34da-4b5c-9cf7-b10e79e7035b/copy-manifests/0.log"
Jul 19 15:26:13.355914 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:13.355884    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6_104c28db-34da-4b5c-9cf7-b10e79e7035b/manager/0.log"
Jul 19 15:26:13.703138 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:13.703047    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-765d74d4c4-mg6lz_deb97b98-9639-492c-b3ef-a55cb48bd128/maas-api/0.log"
Jul 19 15:26:13.813528 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:13.813498    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29741220-5fp44_cce6ed70-fa9b-4372-b0d3-248d8cac5a07/cleanup/0.log"
Jul 19 15:26:14.046096 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:14.046058    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-cvw9l_b95e9338-b3fb-48d2-aae8-d96dd952a32a/postgres/0.log"
Jul 19 15:26:15.468243 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:15.468206    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m9ctk_0dc36132-4e13-41ff-874d-8b2b28a41197/manager/0.log"
Jul 19 15:26:15.589065 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:15.589036    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-xclsw_800dc795-fc6d-47f2-aa71-b45253f15c98/manager/0.log"
Jul 19 15:26:15.825906 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:15.825867    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-frjtx_40b44909-e9ff-4f93-8d34-2be2e7a4779e/registry-server/0.log"
Jul 19 15:26:15.954068 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:15.954035    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bknhk_bfec1624-e981-43b1-bad4-6c043daf08b6/manager/0.log"
Jul 19 15:26:16.195640 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:16.195567    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-875f9_57e5b952-9344-4a2f-8842-0f381ed35815/manager/0.log"
Jul 19 15:26:16.788627 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:16.788592    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc68cb667-699tn_161ce75c-6d40-4d1c-8889-d327ee413665/kube-auth-proxy/0.log"
Jul 19 15:26:17.028382 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:17.028354    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/1.log"
Jul 19 15:26:17.153296 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:17.153211    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-v9dtg_25558de6-bb27-4ff5-816e-a47f98d9e12d/payload-pre-processing/0.log"
Jul 19 15:26:17.282019 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:17.281987    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cfn7l_1e0dc04d-83bd-4da1-8847-9602efff040b/payload-processing/0.log"
Jul 19 15:26:17.746795 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:17.746742    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-n2nzl_3a078fe4-e5b8-4116-ae34-1626cc8fe70e/main/0.log"
Jul 19 15:26:18.203017 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:18.202931    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-g2zdc_d8ed9ab5-3b98-4cf2-8143-f34e66f5bac0/main/0.log"
Jul 19 15:26:24.178177 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:24.178139    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:24.278289 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:24.278257    2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log" limit=1024
Jul 19 15:26:26.966547 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:26.966511    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jjgj4_d62cd493-14c5-4c1c-aef3-ce86ea253b26/global-pull-secret-syncer/0.log"
Jul 19 15:26:27.018961 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:27.018925    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5t7hb_ed8add3f-5454-4a1e-be52-f3131d364a2d/konnectivity-agent/0.log"
Jul 19 15:26:27.162178 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:27.162147    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-105.ec2.internal_3d37fd728dd468b7ee1c820042e75b2d/haproxy/0.log"
Jul 19 15:26:32.113050 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:32.112965    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m9ctk_0dc36132-4e13-41ff-874d-8b2b28a41197/manager/0.log"
Jul 19 15:26:32.173288 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:32.173250    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-xclsw_800dc795-fc6d-47f2-aa71-b45253f15c98/manager/0.log"
Jul 19 15:26:32.250848 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:32.250819    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-frjtx_40b44909-e9ff-4f93-8d34-2be2e7a4779e/registry-server/0.log"
Jul 19 15:26:32.427103 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:32.426879    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bknhk_bfec1624-e981-43b1-bad4-6c043daf08b6/manager/0.log"
Jul 19 15:26:32.648634 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:32.648606    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-875f9_57e5b952-9344-4a2f-8842-0f381ed35815/manager/0.log"
Jul 19 15:26:34.286675 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.286648    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-8zswf_ae2f832f-1cac-4fe5-9697-35a754873776/kube-state-metrics/0.log"
Jul 19 15:26:34.310099 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.310067    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-8zswf_ae2f832f-1cac-4fe5-9697-35a754873776/kube-rbac-proxy-main/0.log"
Jul 19 15:26:34.333349 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.333312    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-8zswf_ae2f832f-1cac-4fe5-9697-35a754873776/kube-rbac-proxy-self/0.log"
Jul 19 15:26:34.389675 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.389644    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-wnckx_7ca1b754-a62e-4629-a90f-20ccdfeecb43/monitoring-plugin/0.log"
Jul 19 15:26:34.432085 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.432003    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68zx8_f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38/node-exporter/0.log"
Jul 19 15:26:34.456877 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.456838    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68zx8_f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38/kube-rbac-proxy/0.log"
Jul 19 15:26:34.482452 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.482426    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68zx8_f73c8d9a-a5f7-481a-950f-ec2d7e2a0c38/init-textfile/0.log"
Jul 19 15:26:34.759132 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.759100    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/prometheus/0.log"
Jul 19 15:26:34.783370 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.783341    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/config-reloader/0.log"
Jul 19 15:26:34.805217 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.805183    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/thanos-sidecar/0.log"
Jul 19 15:26:34.824312 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.824284    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/kube-rbac-proxy-web/0.log"
Jul 19 15:26:34.844612 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.844590    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/kube-rbac-proxy/0.log"
Jul 19 15:26:34.865060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.865039    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/kube-rbac-proxy-thanos/0.log"
Jul 19 15:26:34.884923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.884902    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6778a5d3-9fcf-4c96-bc82-91aba04bf72d/init-config-reloader/0.log"
Jul 19 15:26:34.954726 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:34.954683    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-gkgvs_1f6d96fd-2bdf-404a-a1ea-69df1f9c80b1/prometheus-operator-admission-webhook/0.log"
Jul 19 15:26:36.976005 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.975965    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"]
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976523    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc96968-844d-418c-9821-79f8d2f9c0fa" containerName="revoke-keys"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976545    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc96968-844d-418c-9821-79f8d2f9c0fa" containerName="revoke-keys"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976563    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976572    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976699    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976714    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e59da23b-6f10-4eb5-a632-0e5c358feca5" containerName="istio-proxy"
Jul 19 15:26:36.976797 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.976728    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc96968-844d-418c-9821-79f8d2f9c0fa" containerName="revoke-keys"
Jul 19 15:26:36.980524 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.980508    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:36.982909 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.982884    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9q7mm\"/\"openshift-service-ca.crt\""
Jul 19 15:26:36.983027 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.982886    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9q7mm\"/\"default-dockercfg-8hr7t\""
Jul 19 15:26:36.983920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.983841    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9q7mm\"/\"kube-root-ca.crt\""
Jul 19 15:26:36.987855 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:36.987827    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"]
Jul 19 15:26:37.099706 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.099660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-podres\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.099706 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.099705    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-sys\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.099963 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.099744    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp4qk\" (UniqueName: \"kubernetes.io/projected/4985323b-d963-4dd6-a351-de0932d61368-kube-api-access-cp4qk\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.099963 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.099852    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-proc\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.100046 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.099960    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-lib-modules\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200480 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200432    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp4qk\" (UniqueName: \"kubernetes.io/projected/4985323b-d963-4dd6-a351-de0932d61368-kube-api-access-cp4qk\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-proc\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-lib-modules\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200631    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-podres\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200638    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-proc\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200690 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200655    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-sys\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200710    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-sys\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200759    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-lib-modules\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.200920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.200763    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4985323b-d963-4dd6-a351-de0932d61368-podres\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.209171 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.209148    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp4qk\" (UniqueName: \"kubernetes.io/projected/4985323b-d963-4dd6-a351-de0932d61368-kube-api-access-cp4qk\") pod \"perf-node-gather-daemonset-kzvvj\" (UID: \"4985323b-d963-4dd6-a351-de0932d61368\") " pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.292640 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.292551    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.432994 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.432969    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"]
Jul 19 15:26:37.435089 ip-10-0-141-105 kubenswrapper[2585]: W0719 15:26:37.435058    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4985323b_d963_4dd6_a351_de0932d61368.slice/crio-ee0337143e46a9ac452de820234117c2ba771bc79fb4301c0fa2996b30bef4f9 WatchSource:0}: Error finding container ee0337143e46a9ac452de820234117c2ba771bc79fb4301c0fa2996b30bef4f9: Status 404 returned error can't find the container with id ee0337143e46a9ac452de820234117c2ba771bc79fb4301c0fa2996b30bef4f9
Jul 19 15:26:37.655381 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.655341    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7987948c99-8vqv4_8b12484d-efb2-4a6b-9b69-77d3e451ed9c/console/0.log"
Jul 19 15:26:37.839226 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.839138    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj" event={"ID":"4985323b-d963-4dd6-a351-de0932d61368","Type":"ContainerStarted","Data":"f5b10fab0fbf6b35fa558e6cb88952578f63513f855afb36c1f5365e3aba7cd6"}
Jul 19 15:26:37.839226 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.839189    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj" event={"ID":"4985323b-d963-4dd6-a351-de0932d61368","Type":"ContainerStarted","Data":"ee0337143e46a9ac452de820234117c2ba771bc79fb4301c0fa2996b30bef4f9"}
Jul 19 15:26:37.839413 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.839244    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:37.855167 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:37.855112    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj" podStartSLOduration=1.855094845 podStartE2EDuration="1.855094845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 15:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:26:37.854538516 +0000 UTC m=+3000.281015539" watchObservedRunningTime="2026-07-19 15:26:37.855094845 +0000 UTC m=+3000.281571869"
Jul 19 15:26:38.355580 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:38.355552    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-5zk42_8aafa04c-a08c-4dc2-bd74-8e916c016e45/volume-data-source-validator/0.log"
Jul 19 15:26:38.386448 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:38.386419    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:26:38.396155 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:38.396131    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lrwlt_6c96766b-2429-4fa1-ae33-92005de587c2/ovn-acl-logging/0.log"
Jul 19 15:26:39.402561 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:39.402519    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8lfqd_ca2399d4-f674-4415-aade-10149711f91c/dns/0.log"
Jul 19 15:26:39.426951 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:39.426911    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8lfqd_ca2399d4-f674-4415-aade-10149711f91c/kube-rbac-proxy/0.log"
Jul 19 15:26:39.494960 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:39.494929    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-48n64_cb6b1cf3-bddb-4991-a759-11fb984504be/dns-node-resolver/0.log"
Jul 19 15:26:40.039981 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:40.039947    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5cfdd5c6b5-jbmzd_dbe4bafb-5264-4f7d-92d4-32098cc096fb/registry/0.log"
Jul 19 15:26:40.086396 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:40.086367    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hm8sq_4957289c-6405-4f3f-8531-5099ea2163ae/node-ca/0.log"
Jul 19 15:26:41.234972 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:41.234934    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc68cb667-699tn_161ce75c-6d40-4d1c-8889-d327ee413665/kube-auth-proxy/0.log"
Jul 19 15:26:41.324506 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:41.324476    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/0.log"
Jul 19 15:26:41.375923 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:41.375877    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/1.log"
Jul 19 15:26:41.411060 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:41.411005    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-v9dtg_25558de6-bb27-4ff5-816e-a47f98d9e12d/payload-pre-processing/0.log"
Jul 19 15:26:41.440589 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:41.440560    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cfn7l_1e0dc04d-83bd-4da1-8847-9602efff040b/payload-processing/0.log"
Jul 19 15:26:42.176134 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:42.176094    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6cqn_9565d067-376f-42c4-8fc2-81d72b2898e3/serve-healthcheck-canary/0.log"
Jul 19 15:26:42.719614 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:42.719572    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmgwr_6530713d-747e-4cbe-a95b-f897147db574/kube-rbac-proxy/0.log"
Jul 19 15:26:42.739896 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:42.739864    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmgwr_6530713d-747e-4cbe-a95b-f897147db574/exporter/0.log"
Jul 19 15:26:42.761917 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:42.761893    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmgwr_6530713d-747e-4cbe-a95b-f897147db574/extractor/0.log"
Jul 19 15:26:43.854307 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:43.854282    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9q7mm/perf-node-gather-daemonset-kzvvj"
Jul 19 15:26:44.885283 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:44.885250    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gmvds_d8de0306-00ee-4873-81ed-4306db0c034a/manager/0.log"
Jul 19 15:26:45.204249 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:45.204139    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5khfl_f813f876-72df-41ae-b2cf-a3ad0f598b7b/manager/0.log"
Jul 19 15:26:45.439106 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:45.439069    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qk9cl_c00bb303-832b-484b-87f5-1ffe043c81d6/manager/0.log"
Jul 19 15:26:45.660842 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:45.660809    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6_104c28db-34da-4b5c-9cf7-b10e79e7035b/manager/0.log"
Jul 19 15:26:45.723283 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:45.723252    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b5b7c5f-lcfh6_104c28db-34da-4b5c-9cf7-b10e79e7035b/copy-manifests/0.log"
Jul 19 15:26:46.955967 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:46.955935    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc68cb667-699tn_161ce75c-6d40-4d1c-8889-d327ee413665/kube-auth-proxy/0.log"
Jul 19 15:26:46.998998 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:46.998966    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/0.log"
Jul 19 15:26:47.048675 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:47.048639    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/1.log"
Jul 19 15:26:47.075432 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:47.075395    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-v9dtg_25558de6-bb27-4ff5-816e-a47f98d9e12d/payload-pre-processing/0.log"
Jul 19 15:26:47.098515 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:47.098399    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cfn7l_1e0dc04d-83bd-4da1-8847-9602efff040b/payload-processing/0.log"
Jul 19 15:26:48.397428 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:48.397395    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc68cb667-699tn_161ce75c-6d40-4d1c-8889-d327ee413665/kube-auth-proxy/0.log"
Jul 19 15:26:48.433272 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:48.433237    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/0.log"
Jul 19 15:26:48.490722 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:48.490690    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lrqb5_54f417ba-7683-404b-add7-d30b4d19f264/istio-proxy/1.log"
Jul 19 15:26:48.517688 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:48.517651    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-v9dtg_25558de6-bb27-4ff5-816e-a47f98d9e12d/payload-pre-processing/0.log"
Jul 19 15:26:48.546937 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:48.546905    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-cfn7l_1e0dc04d-83bd-4da1-8847-9602efff040b/payload-processing/0.log"
Jul 19 15:26:51.871449 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:51.871416    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-h6x8z_faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48/migrator/0.log"
Jul 19 15:26:51.890023 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:51.889991    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-h6x8z_faedc00c-ea8b-4eb2-a4c6-7dc4e5f82a48/graceful-termination/0.log"
Jul 19 15:26:52.216196 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:52.216101    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-rzgsz_0805c402-8f62-4c36-9c47-9c66d82625da/kube-storage-version-migrator-operator/1.log"
Jul 19 15:26:52.217773 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:52.217748    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-rzgsz_0805c402-8f62-4c36-9c47-9c66d82625da/kube-storage-version-migrator-operator/0.log"
Jul 19 15:26:53.275503 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.275474    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/kube-multus-additional-cni-plugins/0.log"
Jul 19 15:26:53.294768 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.294741    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/egress-router-binary-copy/0.log"
Jul 19 15:26:53.318312 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.318285    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/cni-plugins/0.log"
Jul 19 15:26:53.337098 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.337072    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/bond-cni-plugin/0.log"
Jul 19 15:26:53.355920 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.355893    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/routeoverride-cni/0.log"
Jul 19 15:26:53.388332 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.388302    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/whereabouts-cni-bincopy/0.log"
Jul 19 15:26:53.441138 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.441104    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h4cnb_b92bec5f-8091-47b8-bfcd-f0d15fec6559/whereabouts-cni/0.log"
Jul 19 15:26:53.729416 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.729386    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q49nn_bd2f75cd-6470-4013-b544-d9f2b9ffdc37/kube-multus/0.log"
Jul 19 15:26:53.794905 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.794874    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vftg8_223f7083-66c8-4efb-9c2e-f19a4b4724be/network-metrics-daemon/0.log"
Jul 19 15:26:53.810146 ip-10-0-141-105 kubenswrapper[2585]: I0719 15:26:53.810119    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vftg8_223f7083-66c8-4efb-9c2e-f19a4b4724be/kube-rbac-proxy/0.log"